# Testing Feedforward Neural Networks Training Programs

Houssem Ben Braiek  SWAT Lab., Polytechnique MontrealMontrealCanada
<EMAIL_ADDRESS>and  Foutse Khomh<EMAIL_ADDRESS>SWAT
Lab., Polytechnique Montréal2500 Chemin de PolytechniqueMontréalCanada

(2020)

###### Abstract.

Nowadays, we are witnessing an increasing effort to improve the performance
and trustworthiness of Deep Neural Networks (DNNs), with the aim to enable
their adoption in safety critical systems such as self-driving cars or
aircraft collision-avoidance systems. Multiple testing techniques are proposed
to generate test cases that can expose inconsistencies in the behavior of DNN
models. These techniques assume implicitly that the training program is bug-
free and appropriately configured. However, satisfying this assumption for a
novel problem requires significant engineering work to prepare the data,
design the DNN, implement the training program, and tune the hyperparameters
in order to produce the model for which current automated test data generators
search for corner-case behaviors. All these model training steps can be error-
prone. Therefore, it is crucial to detect and correct errors throughout all
the engineering steps of DNN-based software systems and not only on the
resulting DNN model.

In this paper, we gather a catalog of training issues and based on their
symptoms and their effects on the behavior of the training program, we propose
practical verification routines to detect the aforementioned issues,
automatically, by continuously validating that some important properties of
the learning dynamics hold during the training. Then, we design,
TheDeepChecker, an end-to-end property-based debugging approach for DNN
training programs and implement it as a TensorFlow-based library. As an
empirical evaluation, we conduct a case study to assess the effectiveness of
TheDeepChecker on synthetic and real-world buggy DL programs and compare its
performance to that of the Amazon SageMaker Debugger (SMD). Results show that
TheDeepChecker’s on-execution validation of DNN-based program’s properties
through three sequential phases (pre-, on-, and post-fitting), succeeds in
revealing several coding bugs and system misconfigurations errors, early on
and at a low cost. Moreover, our property-based approach outperforms the SMD’s
offline rules verification on training logs in terms of detection accuracy for
unstable learning issues and coverage of additional DL bugs.

neural networks, training, testing, debugging

††copyright: acmcopyright††journalyear: 2020††journal: JACM††journalvolume:
XX††journalnumber: XX††article: XX††ccs: Theory of computation Program
verification††ccs: Software and its engineering Maintaining software

##  1\. Introduction

Nowadays, we are witnessing an increase in efforts to improve the reliability
of Deep Neural Networks (DNNs), and consequently the trustworthiness of deep
learning based systems such as self-driving cars or aircraft collision-
avoidance systems (Ben Braiek and Khomh, 2018). Multiple approaches are being
developed to search for test cases that can expose inconsistencies in the
behavior of DNN models; we refer to them as model testing approaches. These
approaches implicitly assume that the trained DNN is already performing well
on the original test dataset compared to known results, i.e., the training
program is bug-free and appropriately configured. However, recent research
works (Islam et al., 2019a) (Humbatova et al., 2020) (Zhang et al., 2018a)
report that bugs exist in DL programs, which invalidates this assumption made
by model testing approaches. Their main finding is that faults that lead to
program crashes represent only a fraction of real bugs found in DNN training
programs. The vast majority of bugs are due to hidden logic errors and
configuration inconsistencies, leading to silent failures that are difficult
to detect (since they do not prevent the program from running and producing a
model). In fact, a DNN is trained using the back-propagation algorithm that
relies on a loss function to estimate the distance between actual predictions
and the ground truth, and then, the estimated error is back propagated through
the DNN’s learnable parameters to adjust their values in the opposite
direction of the loss gradient. In practice, components of the training
algorithm are provided as ready-to-use configurable routines by DL libraries,
however, reusing these routines to implement a training program for a designed
DNN is not straightforward and it can be error-prone. From a fundamental point
of view, the backpropagation algorithm can be considered as a leaky
abstraction since the details of its implementation are not trivial. To
illustrate this point let’s consider the basic rule of weights initialization
which states that values should be small random numbers. Setting weights’
values is not simple and straightforward because the use of dummy random
initialization could prevent the DNN from training. In practice, depending on
developers’ design choices, there is a set of custom weight initializations
that have been formally proven to be optimal choices, and hence should be
adopted by the developers.

DL developers also leverage generic-purpose libraries for scientific
computations and DL routines that support a rich set of APIs, incorporating
many assumptions in order to make them ready-to-use for common problems. These
DL routines can be configured and adapted for new problems. However, their
misuses, i.e., the usage of APIs without fully understanding their inner
functioning and–or checking that their assumptions are fulfilled, are likely
to result in erroneous behavior. The implementation of the training steps can
also contain errors leading to erroneous behavior. It is therefore very
important to test the source code of the training program, to ensure that it
accurately reflects the algorithm’s mathematical formulation.
The current trend in testing the resulting DNN model is not sufficient to
ensure that the training process was bug-free and consequently that the DNN
model is reliable. Because all models are imperfect abstractions of the
reality and subject to residual error, it is not possible to rely on the
observable relations between the inputs and outputs of the model to hunt for
errors in the training program; if the trained model produces a wrong
prediction, it does not necessarily mean that the training program is buggy.
Moreover, the level of performance of a model (in terms of precision, recall,
or accuracy) cannot serve as a reliable predictor of the reliability of its
training process, since it is unclear what should be an appropriate level of
performance for a model, to discard any coincidental high probabilistic
measure while taking into consideration various assumptions regarding model
capacity and data complexity. It is unclear how much precision, recall, or
accuracy may be indicative of errors in the training process. D’amour et al.
(D’Amour et al., 2020) have investigated the credibility of various practical
ML systems, using examples from computer vision, medical imaging, natural
language processing, etc. They conclude the limitation of model testing on the
held-out data, called iid performance, and show that plenty of ML models share
strong iid performance on test datasets, however, they often exhibit
unexpectedly distant behaviors when they are deployed in real-world domains.
Hence, the iid performance test is insufficient to separate between effective
and non-effective models that are trained using, respectively, clean and buggy
training programs. Therefore, we need more indicative measures than
probabilistic correctness measures. It is crucial to develop efficient
debugging techniques that developers can use to detect errors in the training
programs of DNNs. Besides the loss- and accuracy evolution curves, researchers
(Hohman et al., 2018) have recently proposed advanced visual analytics systems
to help DL developers debug and refine the design of their DNNs. Oftentimes,
this requires monitoring individual computational units of the model during
the training, such as activation maps or error gradients produced at each
layer, and after training, an instance-based analysis is performed to identify
misclassified instances from a handful of chosen data instances. However, such
visualization based diagnosis techniques require significant human
intervention and good expertise on deep learning concepts. Given the internal
complexity of a DNN, it is always challenging to select a handful of drawing
representations (i.e., suitable for screen display) that should be watched and
analyzed, interactively. Additionally, these visualizations can hardly be used
automatically to track for regressions after a program update. We believe that
full-fledged software debuggers are needed to support DL developers.
In this paper, we propose TheDeepChecker, the first end-to-end automated
debugging approach for DNN training programs. To develop TheDeepChecker, we
gathered a catalog of fundamental algorithmic and development issues in
relation with the DNN training programs. Then, we inferred the properties that
are violated by these identified training program issues, in order to develop
the verification routines that can be used to detect their occurrences during
the training process. Next, we developed a property-based testing method that
activates the derived verification routines, in order to drive an end-to-end
automated debugging process for DNN training programs. To assess the
effectiveness of TheDeepChecker, we implemented it as a TensorFlow-based
testing framework that enables the automatic detection of the identified
issues in DNN training programs developed with Tensorflow (TF) library. We
rely on the taxonomy of real DL faults elaborated in (Humbatova et al., 2020)
and further searching on Stackoverflow, to identify the structure of faults
occuring in DNN training programs. Then, we inject these faults in clean DNN
training programs to create a set of synthetic buggy programs, each one
containing a particular fault, aiming at challenging the TheDeepChecker in
identifying the faults or steering the users to them by detecting precise
fault-indicative symptoms. Moreover, we assess the performance of
TheDeepChecker on a mixed selection of $20$ real-world TF buggy programs
(Zhang et al., 2018a) splitted equally between snippets of code shared on
StackOverflow (SO) and bug-fixing commits from DL projects hosted on GitHub
(GH). As a DL debugging baseline, we manage to run all the studied buggy DL
programs on Amazon SageMaker Cloud ML service while activating its internal
Debugger’s built-in rules. Results show that TheDeepChecker can successfully
support DL practitioners in detecting earlier a wide range of coding bugs and
system misconfigurations through reported violations of essential DL program’s
properties. Additionally, the comparison with SageMaker Debugger (SMD)
highlights the ascendancy of TheDeepChecker’s on-execution validation of DL
properties over SMD’s rules verification on training logs in terms of
detection accuracy and DL bugs’ coverage.
This paper makes the following contributions.

  * •

We investigate former DL faults’ empirical studies and SO posts about buggy TF
training programs to identify common errors made by DL practitioners and
provide a catalog outlining DL program’s development and configuration
pitfalls;

  * •

We present a property-based DL software debugging approach, which implements
verification routines associated to the discovered DL pitfalls across three
sequential phases: pre-, on-, and post-fitting of the DNN.

  * •

We implement our debugging approach based on TF concepts and features to
demonstrate the effectiveness of our verification routines in detecting real
bugs that occur in TF-based training programs.

  * •

By dissecting the root causes and symptoms of collected DL bugs, we simulate
these bugs in DNN architectures to create a benchmark of synthetic buggy
programs. In addition, a real-world database of buggy DL programs is compiled
from snippets of code shared on SO posts and bug-fixing commits in GH
projects.

  * •

We evaluate the effectiveness of TheDeepChecker in debugging synthetic and
real-world TF programs and compare its detection bugs’ accuracy and coverage
with Amazon Sagemaker Debugger (SMD). Besides, we conduct a usability study on
TheDeepChecker in collaboration with two experienced DL engineers from the
industry, focusing on the usefulness and precision of TheDeepChecker’s
debugging reports to steer users towards finding and fixing the actual faults
in real DL buggy programs.

The remainder of this paper is organized as follows. Section 2 provides
background information about DL, property-based software testing, and
TensorFlow. Section 3 presents common training program pitfalls discovered by
DL researchers and experienced by DL developers. Section 4 introduces our
proposed property-based debugging approach for DNN training programs alongside
their derived verification mechanisms and its TF-based implementation. Section
5 reports about the empirical evaluation of our proposed debugging approach.
Sections 6, 7,and 8 discuss, respectively, the threats to validity, the
limitations and future works, and the related literature. Finally, Section 9
concludes the paper.

##  2\. Background

This section provides background information about DL, property-based software
testing, and TensorFlow.

###  2.1. Deep Neural Networks

A deep neural network (DNN) is an artificial neural network with a stack of
multiple computational layers, hence the adjective “deep”. DNNs are often much
harder to train than shallow neural networks (Glorot and Bengio, 2010).
However, they are endued with a hierarchical features learning that lets them
capture increasingly complex patterns directly from the data when they are
appropriately designed and trained. DNNs include many variants of
architectures that have found success in several domains. We present in the
following the Deep Feed-Forward Neural Network and its popular variant
Convolutional Neural Network.

####  2.1.1. Deep Feed-Forward Neural Networks

Feedforward neural network (FNN) architecture is the quintessential and the
most used neural network (Goodfellow et al., 2016). The objective of FNNs is
to learn the mapping of a fixed-size input (for example, a signal vector) to a
fixed-size output (for example, a probability for each label). Apart from the
input and output layers, FNN contains a cascade of multiple intermediate
layers, which are called hidden layers because the ground truth data does not
provide the desired values for these layers. Last, the name feedforward arises
from the fact that information flows through the processing layers in a feed-
forward manner, i.e., from input layer, through hidden layers and finally to
the output layer. Figure 1 illustrates a simple FNN.

Figure 1. Schema of feedforward neural network

A FNN encapsulates a mapping function $f$ that maps the input $x$ to its
corresponding output $y$ as presented in Equation 1. If $y$ is a category
label, the FNN solves a classification problem and if $y$ is a continuous
value, the FNN performs a regression task.

(1) $y=f(x;W,b)$

Where $W$: weights and $b$: biases represent the learnable parameters.
Given the training data $D=\\{(x_{i},t_{i});\forall i\in[1,n]\\}$, the FNN
training algorithm aims to find the best approximation function $f^{*}$ by
learning the optimal values of its inner parameters $W^{*}$ and $b^{*}$. To do
that, a loss function, $loss(f,x,t)$ is leveraged to measure how well the
prediction $y=f(x)$ matches the ground truth output $t$. Considering the
example of multinomial classification problem, we have $y$, a vector of
probabilities where $y(j)$ represents the probability that $x$ belongs to the
label $j$ and $t$, a one-hot encoding label where $t(j)=1$ if $j$ is the true
label; otherwise $t(j)=0$. The most common loss function is the cross entropy
$loss(f,x,t)=-\sum_{j}{t(j)\log(y(j))}$.
To estimate the loss over all the training data $loss(f,D)$, we use an
expectation, as formulated in Equation 2, that can be either the average or
the sum of data instances’ losses.

(2) $loss(f,D)=E_{i}[loss(f,x_{i},y_{i})]=E_{D}[loss(f,D)]$

Therefore, the optimal parameters $W^{*}$ and $b^{*}$ result in the best
function approximation that spawns the minimum possible estimated loss in the
training data $D$.

(3) $W^{*},b^{*}=\underset{W,b}{\mathrm{argmin}}{E_{D}[loss(f^{*},D)]}$

The loss minimization problem can be solved, iteratively, using gradient
descent algorithm, where the following equations represent an iteration’s
updates :

(4) $W^{(t+1)}=W^{(t)}-\eta\partialderivative{E_{D}[loss(f^{(t)},D)]}{W};\quad
b^{(t+1)}=b^{(t)}-\eta\partialderivative{E_{D}[loss(f^{(t)},D)]}{b}$

As introduced, the FNN assembles multiple computational layers and each layer
$l$ perform a computation; so it encapsulates a kind of sub-function
$f_{l}(x;W_{l},b_{l})$ with inner parameters $W_{l}$ and $b_{l}$. Thus, the
approximate mapping function $f$ of an FNN with $L$ layers is a composite
function formulated as below :

(5)
$f(x;W,b)=f_{L}(f_{L-1}(...(f_{1}(x;W_{1},b_{1})...);W_{L-1},b_{L-1});W_{L},b_{L})$

Where $W=\\{W_{l},\forall l\in[1,L]\\}$ and $b=\\{b_{l},\forall l\in[1,L]\\}$.
Given the huge number of parameters to approximate, the computation of
gradients would be very time-consuming. Deep learning relies on a fast
algorithm, named backpropagation, that applies the derivative chain rule
principle to compute, sequentially, all these derivative backing from the
output to the first hidden layer, while taking full advantage of the
derivatives estimated w.r.t previous layers. Backpropagation is based on two
alternatives main phases, respectively, forward pass and backward pass, which
are detailed below.
Forward Pass. Each hidden layer $l$ contains a set of computation units,
called neurons, that perform a linear transformation $z_{l}$ of their inputs
from previous layers and pass the result through an activation function
$\Phi_{l}$. The latter is a non-linear transformation $a_{l}$ that allows
adding non-linearity in the approximated mapping function in order to be
insensitive to irrelevant variations of the input. The layer’s computation can
be written as:

(6) $z_{l}=W_{l}a_{l-1}+b_{l},\forall l\in[1,L]$ (7)
$a_{l}=\Phi_{l}(z_{l}),\forall l\in[1,L]$

We note that $a_{0}$ which is the input layer activation is equal to the input
data $x$. The hidden layers share generally the same activation function. We
denote them $\Phi_{1}=\Phi_{2}=...=\Phi_{L}=\Phi$. We denote the activation
function of the output layer by $\Psi=\Phi_{L}$.
Backward Pass. First, we introduce an intermediate quantity, $\delta$, where
$\delta_{l}$ is the vector of error associated to the layer $l$. $\delta_{l}$
is computed as the gradient of the loss with respect to the weighted input
$z_{l}$. In the following, we present the equations used by the
backpropagation algorithm to compute the error for every layer. We refer the
reader to the work of Goodfellow et al. (Goodfellow et al., 2016) for the
proof and in-depth details.

(8)
$\delta_{L}=\partialderivative{loss}{z_{L}}=\nabla_{a}(loss)\odot\Psi^{\prime}(z_{L});\quad\delta_{l}=\partialderivative{loss}{z_{l}}=W^{(l+1)T}\delta_{l+1}\odot\Phi^{\prime}(z_{l})$

In Equation 8, $\odot$ is the Hadamard product, which is an elementwise
product of two vectors in a way that for each component $j$, $v\odot u$ means
$(u\odot t)_{j}=s_{j}t_{j}$.
Second, we formulate the gradient of loss w.r.t the DNN parameters using the
computed error term $\delta$.

(9)
$\partialderivative{loss}{W_{l}}=\delta_{l}a^{(l-1)T};\quad\partialderivative{loss}{b_{l}}=\delta_{l}$

Last, we perform both of weights and biases iteration updates in the opposite
direction of their gradients w.r.t the loss.

(10) $W^{(i+1)}=W^{(i)}-\eta\delta_{l}a^{(l-1)T};\quad
b^{(i+1)}=b^{(i)}-\eta\delta_{l}$

In practice, the backpropagation algorithm does not loop over the training
examples and perform the forward and backward passes on each example
separately. Indeed, it relies on mini-batch stochastic gradient descent that
computes both passes on a mini-batch of examples simultaneously. This is done
by formulating the equations presented above as fully matrix-based formulas
given the input matrix $X=[x_{1},x_{2},...,x_{m}]$ of a mini-batch containing
$m$ examples. Thus, the parameters updates are based on the average of loss
gradients estimated over all the examples of the batch, which leads to more
stable gradient updates.

####  2.1.2. Deep Convolutional Neural Networks

Convolutional Neural Network (CNN) represents a particular type of feedforward
network that is designed to process data in the form of multiple arrays, such
as 2D images and audio spectrograms, or 3D videos (Goodfellow et al., 2016). A
CNN contains the following specialized layers that transform the 3D input
volume to a 3D output volume of neuron activations: Convolutional Layer,
Activation Layer, and Pooling Layer.
Convolutional Layer. The main building block of this type of transformation
layer is the convolution. A convolution provides a 2D feature map, where each
unit is connected to local regions in the input data or previous layer’s
feature map through a multi-dimensional parameter called a filter or kernel.
These filters play the role of feature detectors. The feature map is produced
by sliding the filter over the input data, then computing products between the
filter entries and the local input region at each spatial position, to infer
the corresponding feature map response. Different filters are performed in a
layer and resulting feature maps are stacked in 3D volumes of output neurons.
The separate filters aim to detect distinctive local motifs in the same local
regions. However, all units in one feature map share the same filter, because
motifs are generally invariant to location.
Activation Layer. As in any FNN layer, we use an activation function to add
non-linearity to the computed value. The activation layer applies the
activation function on the extracted feature map as an element wise operation
(i.e., per feature map output). The resulting activation map indicates the
state of each neuron, i.e., active or not.
For each hidden neuron $(i,j)$ in a feature map:

(11) $a_{ij}=\Phi(z_{ij});$

Pooling Layer. The pooling layer ensures spatial pooling operation to reduce
the dimensionality of each feature map and to retain the most relevant
information by creating an invariance to small shifts and distortions.
Depending on the chosen spatial pooling operations, it can be average or max
pooling that computes, respectively, the average or max of all elements in a
pre-defined neighboring spatial window size. Therefore, these neighboring
pooling makes the resulting activation maps smaller and robust to irrelevant
variance, which helps shortening the training time and controlling the
overfitting.
Figure 2 shows a typical architecture of CNN with two main stages: (1)
multiple stack of convolution, activation, and pooling that ensure the
detection of relevant features from the input data; (2) the final activation
map is flatten to be a vector of features and is fed to a fully-connected
neural network that performs the prediction on top of these extracted features
to estimate the labels’ scores or the predict value.

Figure 2. Schema of convolutional neural network

###  2.2. Regularization of Deep Neural Network Training

Given the high capacity of DNNs, developing regularization techniques that
prevent the model from overfitting the training data, has been one of the
major research efforts in the deep learning field. In general, regularization
tolerates a relatively small increase of training loss in favor of reducing
the error on the test data with the hope of having a model that generalizes
well on the future coming data.

####  2.2.1. Standard Regularization Techniques

The standard regularization techniques consist in adding restrictions on the
values of the trained parameters, i.e., by adding a penalty term $\Omega(W)$
in the loss function $loss(f^{*},D)$ (see the regularized loss from Equation
12) that can be seen as a soft constraint on the magnitude of parameters to
restrict and smooth their corresponding distributions.

(12) $\tilde{loss}(W,b,D)=loss(W,b,D)+\lambda\Omega(W)$

Where $\lambda$ fixes the relative contribution of the norm penalty omega; so
setting $\lambda=0$ means no regularization and larger values of $\lambda$
correspond to more regularization. The most popular penalty consists in
penalizing the weights of the linear computations; keeping their values closer
to the origin by using L2-norm ($\Omega(W)=\frac{1}{2}\norm{W}_{2}^{2}$)
and–or enhancing the sparsity of the weights by using L1-norm
($\Omega(W)=\norm{W}_{1}$). With the gradient-based learners, we compute the
gradient of the weight penalty term $\Omega(w_{i})$ w.r.t weight $w_{i}$ as
described in the following formula.

(13)
$\frac{\partial\Omega}{\partial\mathbf{w}_{i}}=\left\\{\begin{array}[]{ll}2\times\mathbf{w}_{i}^{(t)},&\text{
if }\Omega(\mathbf{w})=\|\mathbf{w}\|_{2}^{2}\\\
1\times\operatorname{sign}\left(\mathbf{w}_{i}^{(t)}\right),&\text{ if
}\Omega(\mathbf{w})=\|\mathbf{w}\|_{1}\end{array}\right.$

From the above formula, we can see that L2-norm penalty continuously reduces
the magnitude of the weights proportionally to it while L1-norm reduces the
magnitude by a constant. Thus, L2-norm and L1-norm pushes, respectively, the
magnitude of weights towards increasingly lower and zero values. Moreover, the
defined hyperparameter, $\lambda$, controls the strength of the applied
regularization. It is therefore important to adjust it appropriately, taking
into account the design of the model and the complexity of the target problem
but concerning changes w.r.t size of batches, it has been shown that scaling
the $\lambda$ by $1/m$, where $m$ is the size of the batch, can make it
comparable across different size of batches (Andrew Ng and Mourri, 2020),
which avoids tuning manually $\lambda$. Intuitively, the training algorithm
tries to approximate an unknown distribution by minimizing the empirical error
on a sample of data; so a large sample is likely to be more representative of
this unknown distribution, and as a result, less regularization might be
needed in order to capture the maximum information about the target data
distribution.

####  2.2.2. Advanced Regularization Techniques

The advanced regularization techniques for DNNs include normalization and
stochasticity to the DNN inner computations (Garbin et al., 2020). In fact,
manipulating randomly the DNN architecture, over training passes, minimizes
the risks that the learned parameters are highly customized to the underlying
training data. Furthermore, normalizing the input of each layer, not only the
input data, improve furthermore the smoothness of the loss landscape towards
more stable gradient-based optimization, and fortunately, higher chances to
avoid saddle points and ineffective local minima.
Dropout. One of the most frequently used regularization techniques is dropout
(Srivastava et al., 2014), which randomly deactivates a subset of neurons from
the dropped dense or convolutional layer at every training iteration. Indeed,
the degree of randomness of dropout should be adjusted with respect to the
width of the layer using a hyperparameter, $pkeep$, which represents the
neurons’ retention probability. At inference time, dropping neurons is stopped
and compensated by multiplying all the weights in the layer by $pkeep$ in
order to keep the distribution of the layer outputs (i.e., results of the
layer’s affine transformation) during inference time close to the distribution
during training time. Mathematically, for each hidden layer $l$, we define a
binary mask $m^{l}$, where each element can be $1$ with predefined probability
$pkeep$. Thus, the dropped out version of the hidden layer output $z^{l}$
masks out units using element-wise production, $z_{d}^{l}=m^{l}\odot z^{l}$.
Intuitively, dropout reduces the risk of overfitting by making the DNN robust
against the deactivation of some neurons, which forces the DNN to rely on
population behavior instead of the activity of other feature detectors unit
(i.e., preventing the co-adaptation of feature detectors). Indeed, model
ensembling, a well-known technique in statistical learning, consists in
combining the output of multiple models, each trained differently in some
respect, to generate one final answer. The resulting improvements on the
performance metrics explain its domination over recent machine learning
competitions (Goodfellow et al., 2016), however, it requires a much larger
training time by definition (compared to training only one model). More
fundamentally, dropout (Baldi and Sadowski, 2013) simulates model ensembling
without creating multiple neural networks by combining the predictions of
multiple sub-DNNs resulting from dropping, randomly, different subset of
neurons between every two consecutive training passes.
Batch-normalisation. Another interesting regularization strategy designed for
DNN is Batch-normalisation (Batchnorm). It relies on continuously normalizing
intermediate activations across batches during a mini-batch loss minimization
(Ioffe and Szegedy, 2015). Indeed, the proceeded normalization is based on the
standardization of each intermediate feature using the pre-computed mean and
standard deviation on the current batch, respectively,
$\mu_{B}=\frac{1}{m}\sum_{i=1}^{m}x_{i}$ and
$\sigma_{B}^{2}=\frac{1}{m}\sum_{i=1}^{m}(x_{i}-\mu_{B})^{2}$ given the batch
data $B$ of size $m$. The normalized activations
$\hat{x_{i}}=\frac{x_{i}-\mu_{B}}{\sqrt{\sigma_{B}^{2}+\epsilon}}$ would have
zero mean and unit standard deviation. However, the resulting reduction of
magnitude may induce information loss caused by the distortions in the learned
intermediary features, and consequently, the degradation of model’s learning
capacity. Therefore, batch-normalization performs a linear transformation to
scale and shift the normalized activations, $a_{i}=\alpha\hat{x_{i}}+\beta$
with aim of preserving the expressiveness of the DNN through learning
additionally both of the parameters $\alpha$ and $\beta$. In addition, batch-
normalisation also computes two other statistics, $E[x]$ and $Var[x]$, which
represent, respectively, the moving average and the moving variance of the
flowing data during all the training. Thus, at the test time, we use
population mean($E[x]$) and population variance($Var[x]$) to standardize the
layer outputs instead of using batch mean($\mu_{B}$) and batch
variance($\sigma_{B}^{2}$).

Santurkar et al. (Santurkar et al., 2018) investigated the fundamental reasons
behind the effectiveness of batch normalization in regularizing modern DNNs.
They found that normalizing continuously all the inputs of hidden layers
(i.e., all the activations) instead of normalizing only the inputs, makes the
surface of loss smoother, which ensures faster convergence and safer training
using relatively higher learning rates for which unnormalized DNNs (i.e.,
standard variants without batchnorm) diverge. This positive effect on the loss
landscape is called better conditioning of loss minimization problem, which is
described explicitly in the following sub-section.

###  2.3. Conditioning of Loss Minimization Problem

Conditioning refers to how swiftly a function changes with respect to small
variations in its entries (Goodfellow et al., 2016). High sensitive functions
lead to poorly-conditioned scientific computations, such as numerical
optimization. Deep learning algorithms rely on numerical cost minimization.
The performance of DNN is estimated as its prediction ability with respect to
unseen data (which represents future data). However, training a DNN consists
in minimizing the loss on the training data, which means reducing the
empirical error on a sample data with the aim of optimizing indirectly the
true error on the unknown target distribution. Traditionally, machine learning
algorithms design the loss function and constraints carefully; ensuring that
the minimization problem is convex, where any found local minimum is
guaranteed to be a global minimum. When training a deep neural network, the
loss function is not only non-convex but also tends to have a large number of
“kinks”, flat regions, and sharp minima (Li et al., 2018). This makes the
empirical risk minimization difficult. The empirical risk may become non-
representative of the true risk that we aim to reduce. Nevertheless, deep
learning has achieved impressive results by training DNN using first-order
gradient-based optimization. This practical trainability success is highly
dependent on the design of DNN, the choice of optimizer, parameters
initialization, normalization, regularization, and a variety of
hyperparameters. However, finding adequate configurations and parameters can
be very challenging and the optimization of neural networks is still an open
problem. In practice, training a DNN for a novel problem, context or data
requires a series of trial-and-error using different configuration choices and
hyperparameters tuning. These configuration choices have a strong effect on
the conditioning of the minimization problem. In fact, the Hessian matrix of
the loss encapsulates all the second-derivatives that represent the curvature
of the loss surface. The condition number which is the ratio between the
largest and the smallest eigenvalue is very important because a high condition
number indicates a situation of ill-conditioning, where some parameters have
huge curvature while others have smaller one. Such a situation results in a
pathological loss curvature, and a first-order gradient descent would have
difficulty progressing. However, novel DNN design components like skip-
connections of ResNet, advanced regularization and reparameterization
techniques (Bjorck et al., 2018; Ba et al., 2016) have shown an ability to
improve the Lipschitzness of the loss function, which results in the loss
exhibiting a significantly better beta-smoothness. These smoothing effects
impact the performance of the training algorithm in a major way because it
provides more confidence that the estimated gradient direction for each
training step remains a fairly accurate estimate of the actual gradient
direction after taking that step. This enables performing update steps without
high risk of running into a sudden change of the loss landscape; including
flat region (corresponding to vanishing gradient) or sharp local minimum
(causing exploding gradients). In other words, finding good ways to configure
and parametrize the DNN ensures the stability of the loss function and better
predictiveness of its computed gradients.
Li et al. (Li et al., 2018) have shown that sufficiently deeper neural
networks encounter a high risk of sudden transition in their loss landscapes
from being nearly convex to being highly chaotic, which is correlated to the
lack of trainability and dramatic decrease of the generalization error.
Therefore, the conditioning of the minimization problem and the loss landscape
geometry have a substantial effect on the quality of both training and
generalization abilities.

###  2.4. Property-Based Software Testing

Property-based testing (PBT) is a practical testing method (Paraskevopoulou et
al., 2015) that provides a systematic way for reasoning about the properties
of the appropriate program’s behaviors instead of the correct outcomes. For
example, one may validate that a random data generator can produce
probabilities within $[0,1]$, while abstracting away from the actual
probabilities. It follows the philosophy of invariant detection, which defines
a set of invariant properties that allow aligning an incorrect execution
against the expected execution (Ernst et al., 2001). Invariant detection
belongs to the broader class of pseudo-oracle techniques (Barr et al., 2014)
that bypass the lack of genuine oracle and can be used to distinguish a
program’s correct behavior from an incorrect behavior. Thus, DNN training
programs are considered programs that are mainly written to determine the
answer (i.e., the learned parameters). As a result, such programs cannot have
a conventional oracle (Ben Braiek and Khomh, 2018). Invariant detection
belongs to the broader class of pseudo-oracle techniques that bypass the lack
of genuine oracle and can be used to distinguish a program’s correct behavior
from an incorrect behavior. Indeed, PBT defines the essential properties that
the under-test program must respect in any possible execution scenario, rather
than searching for the exhaustive set of all valid input-output pairs. These
properties should represent high-level specifications, describing the
program’s correctness. First, PBT requires the collection of sufficient
properties about the component under test (such as its function, program, or
system). Then, the verification process starts by generating inputs for the
component relying on specific heuristics to cover the equivalent classes of
data inputs. Afterward, it validates for all the generated inputs that all
preconditions, invariant properties on intermediary results, and
postconditions associated to the component under test are totally true. When a
property is failed, the counterexample is shrunk by searching the minimal
combination of input elements that causes the property to fail. In fact, large
inputs may cause the failure of multiple properties; so shrinking the input
allows developers to extract the smallest part of inputs that is capable of
reproducing a particular failure, which is essential for fixing the bug.

###  2.5. TensorFlow Library: Concepts and Features

TensorFlow (TF) (Abadi et al., 2016) is an open source DL library released by
Google to help DL practitioners construct different architectures of DNNs. TF
is based on a Directed Acyclic Graph (DAG) that contains nodes, which
represent mathematical operations, and edges, which encapsulate tensors (i.e.,
multidimensional data arrays). This dataflow graph offers a high-level of
abstraction to represent all the computations and states of the DNN model,
including the mathematical operations, the parameters and their update rules,
and the input preprocessing. Thus, the dataflow graph establishes the
communication between defined sub-computations; making it easy to partition
independent computations and execute them in parallel on multiple distributed
devices. Given a TF dataflow graph, the TF XLA compiler is able to generate an
optimized and faster code for any targeted hardware environment including
conventional CPUs or high-performance graphics processing units (GPUs), as
well as Google’s custom designed ASICs known as Tensor Processing Units
(TPUs). Regarding the running of graph computations, TF follows the principle
of deferred execution or lazy evaluation that consists of two main phases in a
TF program: (1) a construction phase that assembles a graph, including
variables and operations; (2) an execution phase that uses a session to
execute operations and evaluate results in the graph. In our work, we choose
to implement our debugging approach on top of TF because of its high
popularity in the ML community (Braiek et al., 2018).

##  3\. DNN Training Program: Pitfalls

Many issues can prevent the fitting process of a DNN from performing properly,
i.e., finding the best-fitted model. In this section, we elaborate on some of
the common pitfalls in designing and implementing DNNs, while pointing out and
discussing concrete examples of their derived non-crashing bugs.

###  3.1. DL Faults Investigation

Figure 3. Overview of DL Pitfalls Investigation Process

Since deep learning has been increasingly leveraged in diverse real-world
applications, researchers have been interested in studying the software
development challenges for this next generation of software, including the
faults’ taxonomy and bugs characteristics. Zhang et al. (Zhang et al., 2018a)
manually inspected the real-world SO and GH’s Tensorflow programs and
identified some of the DL bugs, their root causes and symptoms at high level.
Then, Islam et al. (Islam et al., 2019b) extended the investigation by
including DL programs written with other competitive libraries such as Pytorch
and Caffe, and studied furthermore the categories of bugs and their
relationships. More recently, Humbatova et al. (Humbatova et al., 2020)
refined the former bug investigation (Zhang et al., 2018a; Islam et al.,
2019b) into a taxonomy of real faults that occur in DL software systems. The
taxonomy was deduced from 375 labeled buggy DL code examples built using three
popular DL libraries: Tensorflow, Keras and Pytorch. Moreover, the
construction of the taxonomy was built in collaboration with 20 DL developers
and validated by a different set of 21 DL developers who confirmed the
relevance and completeness of the identified categories. Indeed, a bunch of
the reported bugs were caused by either coding mistakes, model design issues,
or wrong configurations, that share a common high-level symptom of inefficient
training. The latter manifests through convergence difficulties, preventing
partially or even totally the training program from fitting the data. Hence,
these non-crashing bugs are unique to the deep learning software systems that
do not raise exceptions but adversely affect the training dynamics and
results. Thus, we aim to identify and understand the development pitfalls and
root causes behind the non-crashing DL bugs. First, we started by filtering
them from the DL faults collected and reported in the former studies’
datasets. Mainly, we discarded the two categories of Tensors&Inputs (Humbatova
et al., 2020) and GPU usage (Humbatova et al., 2020) that represent,
respectively, crash-inducing bugs and GPU-related bugs. We focus on detecting
the non-crashing bugs among the remaining three categories of Model (Humbatova
et al., 2020), Training (Humbatova et al., 2020) and API (Humbatova et al.,
2020) that contain, respectively, different misconceptions of the model,
multiple poor coding/configuration bugs in the training algorithm
implementation, and misuses of the DL libraries’ API. Then, we extend the
selected subset of bugs with more Q&A posts from Stackoverflow that are
related to this family of bugs. We conduct a keyword-based search on
StackOverflow (SO) with queries in the form of ’bug_type-related
keywords+Tensorflow’ and we select, for each bug type/query, the top-100 SO
posts (sorted by SO internal relevance criterion). Next, we inspect manually
the SO post content including the shared code snippets and users’ comments,
with the aim of identifying more instances of the studied faults in
Tensorflow. Therefore, we found $155$ bug reports in relation to occurrences
of our targeted DL faults in Tensorflow DNN programs. Overall, only $8\%$ of
the bugs in the three above-mentioned categories from (Humbatova et al., 2020)
are included directly into our dataset, but as we searched using their
keywords on SO, $38\%$ of them represent same bug types/root causes in our
datasets. In fact, these buggy DL programs could not be included because they
are either related to another DL framework (Pytorch, Keras, etc.) or missing
necessary information for reproduction such as training examples or
hyperparameters’ values.

Figure 4. Distribution of the collected Tensorflow Bugs over the Taxonomy of
DL Faults (Humbatova et al., 2020)

Figure 4 shows the categories and subcategories of the bugs we collected in
our datasets with respect to the taxonomy of DL faults (Humbatova et al.,
2020). During the manual inspection of these bugs’ instances, we abstract
their main root causes and their typical symptoms (i.e., the negative effects
observed on the training dynamics and the produced DL model), relying on the
practitioners’ shared content on troubleshooting and debugging the DL training
algorithms including blog posts (Ivanov, 2017; Shao, 2019; Bansal, 2020;
Deeplearning4j, 2019; Karpathy, 2020) and popular forum discussions
(community, 2018; Karpathy, 2018a). Indeed, the decontextualization of DL bugs
from the leveraged API version, the used data and the targeted application,
allows recognizing firstly the design or implementation pitfalls that can be
the origin of these training issues. Lately, the concrete occurrences of DL
pitfalls would serve us in the creation of the synthetic buggy examples
(section 5.1.4), which have been used to evaluate the effectiveness of our
debugging approach on detecting the targeted DL-specific bugs. Figure 3
illustrates the schema of the above-mentioned steps to systematically enrich
the datasets of non-crashing bug reports, as well as, identify their main root
causes and symptoms. In the following, we present a comprehensive review of
the DNN training pitfalls, organized in groups based on the main problematic
component of the DNN training program.

###  3.2. Input Data-related Issues

A DNN training program implements a data-sensitive algorithm whose inner logic
is learned from the training data and generalized to future unseen data. Poor
training data quality often translates into an unstable and inefficient
training process. Below, we detail the training issues in relation to the
input data and DNN components making use of them.

####  3.2.1. Unscaled Data

The scale of DNN inputs and outputs (LeCun et al., 1998) is an important
factor that affects the quality of the training. In fact, larger scale input
features produce larger intermediate activations, and consequently, larger
gradients regarding the weights connected to these over-scaled input features
compared to others. Similarly, over-scaled predicted quantities would generate
larger errors and gradients. Inversely, an abuse of data re-scaling penalizes
the quantities with initially a small range of values. Both situations will
induce a pathological loss curvature and an ill-conditioned loss minimization
problem. As a result, the risk of gradient unstable phenomenon (Grosse, 2017)
increases. Modern deep neural networks deploy inner normalization techniques
(Ioffe and Szegedy, 2015; Ba et al., 2016) to overcome unstable distributions
of the computed activations and gradients, however, their optimization
routines should cope with high update oscillations during the early stage of
training because of the unscaled data, and as a result, the DNN is firstly
trained on how to scale and shift intermediate calculations into an
appropriate range. This overhead complexity slows down the training procedure
and might prevent the convergence towards the best-fitted model.

####  3.2.2. Distribution-Shifting Augmentation

Given their high learning capacity, DNNs require relatively large and
sufficient training data to avoid simply overfitting the data. Since many
application domains lack the access to big data and because gathering data is
expensive, DL developers often resort to data augmentation techniques (Perez
and Wang, 2017) to increase the quantity and diversity of their training data.
Examples of data augmentation techniques for images include geometric
transformations, color space augmentations, kernel filters, random erasing,
and cropping. Nonetheless, the use of inappropriate augmentation rules, as
shown in the SO posts #57275278, #48845354 and #55786384, can induce a shift
in the training data distribution that prevents the DNN from learning
effectively. A DNN trained on noisy, shifted data is often hard to converge to
a stable state and also incapable of predicting correctly on unseen data
(i.e., validation or testing datasets).

####  3.2.3. Corrupted Labels

The data used for training supervised machine learning problems are composed
of features $X$ (predictor inputs) and labels $y$ (supervised outputs to
predict). The DNN’s loss minimization problem is non-convex with several
possible local minima; so standard gradient descent often falls into those
minima because of the unchanged input data $X$ over all the training
iterations. To overcome this problem, a mini-batch gradient descent with
shuffling has been used to train DNNs, as introduced in 2.1.1. Indeed,
shuffling the data instances and performing the gradient estimation on only a
subset of them, makes the batch inputs $X_{b}$ change with every iteration.
This helps the optimizer to avoid sub-optimal local minima with relatively
noisy and frequent updates to the DNN’s parameters. In the implementation, we
handle the features $X$ and labels $y$ in separate data structures because the
labels should be used only for estimating the loss and performance metrics in
supervised learning problems. A common bug in the data shuffler or mini-batch
loader, as reported by SO users in #47866803, #46136553 and #41864333,
consists of inducing mistakenly a mismatch between features and labels. This
represents a particular case of a more general DL issue of corrupted labeled
data, which has been extensively studied in the machine learning community
(e.g., (Van Rooyen and Williamson, 2017)).

####  3.2.4. Unbalanced Data

Very often in classification problems, there is an unequal number of instances
for different labels. An unbalanced dataset biases the predictions towards the
majority class or group of labels. Various mitigation techniques have been
proposed to address this issue, e.g., over-sampling, under-sampling, or
weighted loss function (More, 2016). DL developers should be aware of this
situation, whenever it exists, in order to use earlier a mitigation technique
for class imbalance or improve the performance measure to capture fairly
mispredictions for underrepresented classes. Otherwise, a biased model could
be selected based on the overall accuracy among all classes, resulting in
erroneous or unfair behavior when dealing with instances that belong to ones
that are underrepresented.

###  3.3. Connectivity and Custom Operation Issues

To implement a DNN training program, DL developers use DL libraries that allow
constructing the computational graph, where nodes and edges represent,
respectively, operations and data paths. The operations represent the
computational units that form the linear computations, activations, and
gradient estimations. Data paths interconnect the operations and allow data to
flow from one operation to the next, in order to successfully train and use
the model. Through program code, DL developers use library’s built-in and
newly-implemented components for operations and connect them by either feeding
one’s outputs as inputs to another or by performing a math operation joining
them. Configurable routines enable rapid development and expansion of reliable
DNN programs, however they may lead to spaghetti code that becomes too large
with scatter variables and glue code (build bindings between components),
increasing the risk of coding errors.

####  3.3.1. Network Disconnections

The most basic dependency is between the inputs and the outputs of the DNN.
The DNN should predict the outputs based on the information distilled from the
inputs. Thus, a DNN training program that does not consider the inputs when
performing its internal computations is definitely erroneous. Moreover,
disconnections can occur between the intermediate layers. Indeed, DL engineers
can forget to connect some branches of the DNN or to pass the right inputs to
the layers. When such omissions occur, one or more DNN layers are accidentally
removed. A DNN with fewer layers than necessary can still be trained. A DNN
can converge to an acceptable performance with only partial layers. If this
occurs, however, the program will no longer comply with its specifications and
its performance may be severely impacted. An illustrative example (Roberts,
2018) of a connectivity bug occurs when cloning multiple times the code block
for constructing a layer, the DL developer may forget to change the input and
output for one of these constructed layers which makes it disconnected from
the neural network.

####  3.3.2. Incorrect Custom Operation

The common abstractions used by computational units to encode numerical data
are tensors, which are multidimensional arrays with supported algebraic
operations. These tensors make it easy to manage high dimensional parameters
and perform operations on them efficiently. However, the translation of math
formulas from scientific pseudo-code to tensor-based operations can be error-
prone. As an illustration, let’s consider the cross-entropy loss which is a
matrix-matrix operation that accepts the probabilities matrix and the matrix
of one-hot encoding labels in order to estimate a particular distance. A buggy
loss function may not correctly broadcast the operation if the reduction is
done over the wrong axis (e.g., sum over rows instead of columns) and mix
information between independent data instances of the batch. This introduces
an incorrect dependency to the loss function. This issue can be difficult to
detect since the DNN can still train and converge poorly and in the best case,
can learn to ignore data coming from other batch elements. Besides, DL
libraries include an automatic differentiation module that generates the
analytical formula and computes the gradient automatically. However, DL
developers can include non-differentiable or problematic operations in their
custom function as shown in the SO posts #41780344 and #54346263, which
negatively affect the gradients flowing over the newly-designed DNN. In
similar way, DL developers can also hand-crafted the gradient calculation for
their custom operations, but these gradients implemented from scratch should
be tested carefully to avoid wrong computations as motivated by the SO posts
#46876063 and #64172765.

###  3.4. Parameters-related Issues

DNN parameters represent the weights and biases of a DNN’s layers. These
parameters are randomly initialized, then, they are optimized during the
training process. In the following, we discuss pitfalls in the initialization
of parameters that can affect their learning dynamics.

####  3.4.1. Poor Weight Initialization

An improper initialization of the weights for a DNN hampers the stability of
the learning optimization problem, leading to unstable activation during the
forward pass and unstable loss gradients of the backward flow. First, the
constant weights induces a symmetry between hidden neurons of the same layer.
Thus, the hidden units of the same layer share the same input and output
weights, which makes them compute the same output and receive the same
gradient. Hence, each layer’s neurons perform the same update and remain
identical; i.e., wasting capacity. Second, random sampling of initial weights
breaks the symmetry between the neurons, however, the quality of training is
strongly affected by the choice of initialization (Goodfellow et al., 2016).
Indeed, the derivative equations 8 and 9 show that the estimated gradients
include multiplication by weights, which makes their initial magnitude scale
affects their growth or decay over iterations and might induce exploding or
dead weights (Evci, 2018).

####  3.4.2. Ineffective Bias Initialization

A bias is like the intercept added to a linear equation. Its main purpose is
to allow degrees of freedom close to the origin, which improves the
representation capacity of a neural network; so it can fit better to the given
data. Generally, the initial biases are always set to zeros. Despite this,
null bias for particularly-skewed data distributions (e.g., unbalanced
datasets) slows down DNN training, which would do the bias calibration during
its first few iterations. It means that non-zero bias could contribute
significantly to fit the model if it is delicately set up to approximate the
bias of data. For example, learning a classification problem with a rare label
is a kind of bias already known; so the final layer’s bias should be carefully
initialized to accelerate the learning task.

###  3.5. Activation-related issues

Activation represents the intermediate computation that introduces non-
linearity to filter the information computed by the previous layer. In the
following, we discuss some problems related to activations.

####  3.5.1. Activations out of Range

Activation functions are nonlinear functions that determine the output of a
neural network. The function is attached to each neuron and determines whether
it should be activated (“fired”) or not, based on the relevance of neuron
output for the model’s prediction. Activation functions also help normalize
the output of each neuron by transforming inputs into outputs that are within
a predefined range of values. When DL developers implement an activation
function from scratch, there is a risk of bugs that leads to a wrong or
unbounded mathematical function yielding outputs within a range inconsistent
with what is expected by the developer (e.g., sigmoid’s outputs are between
$[0,1]$ and tanh’s outputs are between $[-1,1]$).

####  3.5.2. Inadequate Hidden Layer Activation

Although the choice of hidden activation function is a design engineering
problem, it is not an empirical and performance-driven selection because there
are activations that are more suitable and even specialized for particular use
cases rather than others. In fact, a non-linear activation is an essential
design component; so a bottom-line inadequate choice would be keeping identity
function for activation as evidenced by SO posts #53138899 and #46181692.
Nevertheless, softmax is a special non-linear activation designed specially to
transform the logits into probabilities; so mistakenly choosing it, as an
activation for hidden layers, would likely hinder the parameters learning and
often discourage a smooth flowing of gradient, as shown in the SO post
#52575271. Below, we detail training issues in relation with well-known hidden
activations that could happen in certain design circumstances, where selecting
an alternative activation function should be considered.

##### Saturation of Bounded Function

Activation functions with a bounded sigmoidal curve, such as sigmoid or tanh,
exhibit smooth linear behavior for inputs within the active range and become
very close to either the lower or the upper asymptotes for relatively large
positive and negative inputs. The phenomenon of neuron saturation occurs when
a neuron returns only values close to the asymptotic limits of the activation
functions. In this case, any adjustment of the weights will not affect the
output of the activation function. As a result, the training process may
stagnate with stable parameters, preventing the training algorithm from
refining them. In fact, we can write the equation index-free to illustrate the
gradient computation flow in general:

(14) $\partialderivative{loss}{W}=a_{in}\times\delta_{out}$

where $a_{in}$ is the activation of the neuron input to the weight $W$ and
$\delta_{out}$ is the error of the neuron output from the weight $W$.
When the activation function $\Phi$ is saturated, its outputs are in the flat
region where $\Phi^{\prime}\approx 0$; so $\delta_{out}\approx 0$ and $W$
freezes or learns slowly.

##### Dead ReLU Function

ReLU stands for rectified linear unit, and is currently the most used
activation function in deep learning models, especially CNNs. In short, ReLU
is linear (identity) for all positive values, and zero for all negative
values. Contrary to other bounded activation functions like sigmoid or tanh,
ReLU does not suffer from the saturation problem because the slope does not
saturate when $x$ gets large and the problem of vanishing gradient is less
observed when using ReLU as activation function. Nevertheless, ReLU risks
“dead ReLU” phenomenon (Evci, 2018) because it nullifies equally all the
negative values. A ReLU neuron is considered “dead” when it always outputs
zero. Such neurons do not have any contribution in identifying patterns in the
data nor in class discrimination. Hence, those neurons are useless and if
there are many of them, one may end up with completely frozen hidden layers
doing nothing. In fact, given the index-free Equation 14, we can see that when
the activation is zero $a_{in}=0$, the loss gradient w.r.t weights becomes
zero too ($\partialderivative{loss}{W}=0$); therefore $W$ freezes and no
longer receives updates. This problem is often caused by a high learning rate
or a large negative bias. However, recent ReLU variants such as Leaky ReLU and
ELU are recommended as good alternatives when lower learning rates do not
prevent this issue.

####  3.5.3. Inadequate Output Layer Activation

Concerning the output layer, the activation function should map the internal
calculated results into valid predictions. In case of mismatch between the
ranges of last activation layer’s outputs and ground truth labels, the model
could not learn a correct mapping function since it is not able to produce the
full range of possible outcomes.

##### Classification Outputs

The model is learning to predict probabilities, so sigmoid and softmax are the
best candidates for, respectively, binary and multinomial classification. For
instance, a missing softmax layer prior to the cross-entropy calculation can
lead to performance degradation and numerical instability issues as evidenced
by the SO post #53254870. However, the use of softmax for a classifier model
with $1$-dim output leads to the incapacity of outputting the full range of
class labels, as evidenced by the SO posts #59129802 and #53971451 where
sigmoid should be used to output the negative class $0$, or #51993989 where
tanh should be used to output both of labels: $0$ and $-1$. Other common
pitfalls are stacking consecutive output activations, which add useless
computation levels that may erase relevant learned information, obstruct the
natural gradient flow, and adversely affect DNN performance. Indeed, the
redundancy activations were often result from: (1)API misuse, where recently-
provided stable API loss functions with the logits activation, softmax or
sigmoid, included (i.e., tf.nn.softmax_cross_entropy_with_logits and
tf.sigmoid_cross_entropy_with_logits), mislead several DL practitioners that
passed the result of last layer activation to these loss functions, which
resulted in a double application of softmax or sigmoid on the outputs (e.g.,
we refer to SO posts #36078411, #46895949, and #42521400); (2)misconception of
abstractions, the definition of a function that abstracts the creation layers
with some parameters to facilitate stacking the neural network’s layers,
however, useless non-linear activation can be applied to the last layer before
probabilities transformation by mistake, which restricts the range of outputs.
For instance, a ReLU activation before applying softmax, as happened in the SO
post #44450841, would nullify negative values and make all their corresponding
labels share the same probability after applying the softmax.

##### Regression Outputs

The last activation should map the internal computations into a range of
values that equals (or is the closest) to the actual interval of target
outputs, in order to ease the optimization process. For instance, SO posts
#60801900 and #64998875 show how the use of Relu, having an output range of
$[0,+\infty]$, prevents the estimation of negative targets and the SO user in
the post #62313327 should switch from sigmoid (i.e., outputs values within
$[0,1]$) to tanh (i.e., outputs values within $[-1,1]$) to meet the real range
of ground truth labels.

####  3.5.4. Unstable Activation Distribution

The activations encode the representation of features detected at that
processing layer during the training process. Thus, the fired activations
indicate that the DNN already detects low-level features, which could be
relevant for following layers. That is why the stagnation of activations
caused by saturation or dead phenomenon hinders the capacity of the DNN to
learn useful patterns from the data. Similarly, over-activated layers that are
active for all inputs and unstable activation layers that have high
variability in their values can lead to numerical instability and–or
divergence problems. In fact, activations represent the input features of the
next layer. The internal computation of this layer adjusts the parameters in
order to infer patterns from features (i.e., activations).The internal
computation of this layer adjusts the parameters in order to infer patterns
from features (i.e., activations). By analogy to the input normalization, the
distribution of the intermediary detected features (inside the DNN) is
important to ensure an effective optimization using backpropagation of loss
gradient through layers’ parameters. More formally, the index-free partial
derivative formula 14 shows well how the magnitude of activations affects
directly the magnitude of weight updates. Researchers (Ba et al., 2016) (Ioffe
and Szegedy, 2015) have proposed different techniques to normalize the outputs
of hidden layers and obtain activations with zero mean and unit standard
deviation. Concretely, these additional internal scaling transformations are
important to control the magnitude of the gradients and improve, formally, the
$\beta$-smoothness and the Lipschitzness of the estimated loss.

###  3.6. Optimization-related issues

The optimization of DNN’s learnable parameters consists in minimizing,
iteratively, the loss, i.e., empirical error of DNN’s predictions regarding
supervised training data. Actually, gradient-based algorithms such as SGD,
Momentum, and Adam, are the preferred way to optimize the DNN’s internal
parameters. Next, we discuss several issues that impede the optimization
process, while describing our proposed verification routines to catch them
earlier.

####  3.6.1. Wrong or Inappropriate Performance Measurements

The iterative optimization of parameters often converges to an equilibrium
behavior of the DNN. At this point of equilibrium, the optimal or near-to-
optimal DNN status is reached. To find this best-fitted DNN (i.e., highest
accuracy or lowest absolute error), the DNN training algorithm acts indirectly
by minimizing a loss function estimated on the training data with hope of
improving the performance of the on-training DNN. Hence, the loss is primarily
designed to measure the distance between predictions and real outputs, while
it should respect fundamental properties of an objective function for first-
order gradient optimization. Empirical loss minimization for DNN training
works well when the minimized loss represents the fitness of the DNN relative
to the data. Thus, a wrong loss function with regards to true model risk,
misleads the training algorithm that, despite its success to reduce the loss,
could not improve the target performance measure (e.g., classification
accuracy). For instance, inadequate choice of loss function like choosing mean
squared error (MSE), which is a standard loss for regression problems, to
compute the deviation between predicted probabilities and target class in a
classification problem (e.g., SO posts #38319898 and #50641866). Another
common fault in relation with the loss is the use if ineffective loss
reduction strategy like in these SO posts where there are no reduction at all
(#36127436) or a sum instead of mean reduction (#43611745 and #41954308).
Indeed, the reduction strategy allows to aggregate the losses computed for all
of the data instances into a scalar loss value. The aggregation could be the
average or the sum, however, mini-batch gradient descent variants are commonly
used for minimizing the DNN’s non-convex loss function. Hence, mean reduction
is better than sum reduction, because averaging losses over the mini-batch
would keep the magnitude of loss independent of the batch size and of other
hyperparameters that are also sensitive to the magnitude of loss gradients
like the learning rate. Besides, a mistaken performance metric also regresses
the expected covariance between both training quality measures (e.g., loss and
accuracy). For instance, bad choice of accuracy metric with respect to the
problem would yield illogic performance measurements and it can be the use of
classification accuracy rate for a regression problem, the use of multiclass
accuracy metric for a binary classifier, or inversely, the use of binary
classification accuracy metric for a multinomial classifier, as evidenced by
the SO posts respectively, #62566558, #62354952 and #42821125.

####  3.6.2. Inadequate Learning rate

As shown in the update equations 10 for weights and biases, the predefined
learning rate controls the magnitude of update at each step; so setting the
learning rate too high or too low can cause drastic changes to the
optimization process and cause several erroneous behaviors. A high learning
rate would push the layer’s parameters changing rapidly in an unstable way;
preventing the model from learning relevant features. The intuition is that
the parameters are a part of the estimated mapping function, so we risk
overfitting the current processed batch of data when we try to strongly adapt
the parameters in order to fit this batch. An excessively-high learning rate
may lead to convergent loss minimization and numerical instability by having
NaN loss or output values. Inversely, a low learning rate can slow down the
parameters changing; making it difficult to learn useful features from data,
and consequently, the minimization process may not converge to a steady state
and may even experience a non-decreasing loss value during training. Starting
by ineffective learning rates is very common when the DL developer is dealing
at first time with a learning problem as evidenced by these SO posts
#42264716, #62381380, #55718408, #34743847, #47245866, #40156629 and
#59106542.

####  3.6.3. Unstable Gradient Problem

The loss gradient equations 8 and 9 show that the gradient of a layer is
simply the product of errors back-propagated from all its next layers (i.e.,
following the forward direction). Intrinsically, the layers tend to learn at
different speeds and deeper neural networks can be subject to unstable
situations if no advanced mechanism is applied to balance out the magnitude of
gradients. However, the unstable gradient problem could be more severe and
could manifest in the form of vanishing or exploding gradients, as described
below, due to poor design choices of initializations and hyperparameters.

##### Vanishing Gradient

In this case, the gradient tends to have smaller values when it is back-
propagated through the hidden layers of the DNN. This causes the gradient to
vanish in the earlier layers, and consequently, it would be nullified or
transformed to undefined values such as Not-a-Number (NaN) caused by underflow
rounding precision during discrete executions on hardware. The problem of
vanishing gradient can lead to the stagnation of the training process and
eventually causing a numerical instability. As an illustration, we take the
example of a DNN configured to have sigmoid $\sigma$ as activation function
and a randomly initialized weight using a Gaussian distribution with a zero
mean and a unit standard deviation. The sigmoid function returns a maximum
derivative value of $\sigma^{\prime}(0)=0.25$ and the absolute value of the
weights product is less than $0.25$ since they belong to a limited range
between $[-1,1]$. Hence, it is apparent that earlier hidden layers (i.e.,
closer to the input layer) would have very less gradient resulting from the
product of several terms that are less or equal to $0.25$. Therefore, earlier
layers receiving vanishing gradients would be stagnant with low magnitude of
weights’ changes.

##### Exploding Gradient

The exploding gradient phenomenon can be encountered when, inversely, the
gradient with respect to the earlier layers diverges and its values become
huge. As a consequence, this could result in the appearance of $-/+\infty$
values. Returning to the previous DNN example, the same DNN can suffer from
exploding gradients in case the parameters are large in a way that their
products with the derivative of the sigmoid keep them on the higher side until
the gradient value explodes and eventually becomes numerically unstable.

Therefore, advanced mechanisms like batch (Ioffe and Szegedy, 2015), layer (Ba
et al., 2016), weight (Salimans and Kingma, 2016) normalizations and tuning of
optimization hyperparameters such as learning rate or momentum coefficients,
are needed to provide adaptive gradient steps and to establish relatively
similar learning speed for all the neural network’s layers.

###  3.7. Regularization-related issues

The regularization strategy prevents the model from overfitting the data,
while allowing the DNN to acquire enough learning capacity to learn useful
patterns and fit the data properly. In the following, we introduce potential
issues related to incorrect and ineffective regularization techniques.

####  3.7.1. Lack of-or-incorrect Regularization

Regularization techniques (Goodfellow et al., 2016), including penalty cost on
the weights magnitude and specialized DL regularization like dropout,
discourages the optimization from exploring complex models and exploiting
spurious correlations in favor of reducing further the loss. Thus, lack of
regularization leads to a noiseless training process with high capacity
modeling that risks capturing even residual variations in the given sample of
data and tends to overfit it quickly. Concretely, as shown in Equation 12, a
zero or very low $\lambda$ makes the penalty cost useless with no effect on
the objective loss function, which enables the free-growth of weights and
intensifies the threat of overfitting even coincidental noises in the sampled
batches used for training the parameters. Regarding dropout, a high retainment
probability of neurons ($pkeep$) for wide dense layers or large activation
maps for convolutional layers decreases the size of the dropped subset of
neurons, which eliminates the randomness effect introduced between the input
features at each inference calculation. Concerning batchnorm, a common mistake
(e.g., SO posts #43234667 and #52279892) consists in forgetting or misusing
the routine that ensures the continuous estimation of the moving average
$E[x]$ and variance $Var[x]$ during the training. Moreover, batchnorm makes
the loss function dependent on the batch size because an instance of the batch
can affect the batch mean and variance estimated, and consequently, affect
both activations and loss values for other instances in the batch. For
example, the use of unit batches, as reported in the SO post #59648509, should
not be applied because the batch variance would be zero and relatively small
batches would increase randomness and make the statistics estimation noisy.
Thus, the DNN fails to perform any normalization on the intermediary
calculation results at the inference time, which means that the regularized
version of DNN becomes incorrect and its inner calculations are non-
representative. In case of mixing dropout and batchnorm, Li et al. (Li et al.,
2019) reported a disharmony issue, named variance shift, between dropout and
batchnorm when applying dropout first. Indeed, the population statistical
variance estimated by batchnorm on the entire DNN training becomes
inconsistent and non-representative because of the shift variance of weights
done by the dropout when the DNN is transferred from the training to the
testing mode. In other words, dropout proceeds by randomly removing the
information coming from a subset of neurons to prevent possible neurons’ co-
adaptation. Thus, we have to pass the cleaned information (i.e., after
dropping out some neurons) through batchnorm statistics estimator, otherwise,
the statistics would be biased by considering all the neurons, i.e., dropped
ones included. A common symptom for all the above issues is that
validation/testing error rates are higher than training error rates. However,
this symptom is quite connected to overfitting situations; so more fine-
grained symptoms are necessary to guide the users towards the occured issue.

####  3.7.2. Over-Regularization

Inversely, a too strong regularization (with high $\lambda$) can significantly
reduce the magnitude of weights, which may result in underfitting; leading to
useless, dead weights, as discussed in the SO posts #51028324 and #51028324.
Regarding dropout, a low $pkeep$ will be considered as a strong regularization
because it introduces too much randomness and may prevent the DNN from
convergence to a stable state; so $pkeep$ should be tuned carefully
considering the underlying DL problem and the depth of the designed neural
network. For instance, many SO posts #60591577, #44832497, #46515248,
#44695141, and #64289118 reported poor training convergence and model
performance resulting from inappropriate configuration of dropout layers.
Indeed, it has been shown that $pkeep$ cannot go below the minimum of $0.5$,
which represents the maximum additive regularization (Srivastava et al.,
2014). In practice, the fixed $pkeep$ for hidden layers, and especially,
layers close to the output layer, is recommended to be within $[0.5,0.8]$.
However, $pkeep$ for the input layer should be kept to about $0.8$ or higher
(i.e., closer to $1.0$). In fact, during the test time, to compensate for
disabling the dropout, the learned parameters are scaled by the $pkeep$
factor. However, Gal et al. (Gal and Ghahramani, 2015) has shown that this
approximation at test time becomes noisier and less accurate as the underlying
layer is far from the output; which can explain the high risk of instability
induced by low neurons’ retainment probability for the earlier layers,
especially, convolutional layers in CNN and the input layer. Therefore, the
common effect of strong regularization is having low error rates for
training/validation/testing data, however, this indicates an underfitting
situation in general. In our debugging methodology, we propose verification
routines to target accurately these over-regularization issues.

##  4\. DNN Training Program : Property-Based Debugging Approach

In the previous section, we have presented DNN training pitfalls related to
misconfigurations and coding bugs, organized by problematic components, to
comprehend their symptoms and their negative effects on the dynamics of
training and the quality of optimization. In this section, we introduce an
adaptation of property-based software testing (PBT) that assembles the
verification routines that we propose for debugging DNN training programs.

###  4.1. Property-Based Model Testing

It has become mainstream research to focus on DL testing approaches that
explicitly check for model properties. Regarding security (Bae et al., 2018)
and robustness (Katz et al., 2017), the learned function should be Lipschitz,
such that small perturbations to the input are guaranteed to spawn bounded
changes to the output. In a similar way, other important properties have been
proposed to satisfy the desired requirements in terms of privacy (Bae et al.,
2018) and fairness (Aggarwal et al., 2019). Indeed, the validation of a
trained DL model’s properties is quite similar to traditional property-based
testing (PBT) because it consists of verifying that the implemented/trained
deterministic function, mapping the inputs to the outputs, satisfies the
desired properties for all the valid inputs. However, the straightforward
application of PBT to the DNN training program, which is a stochastic data-
driven optimization algorithm, appears to be quite challenging. In fact, the
DNN training consists of a data-driven, iterative process guided by the
backpropagation equations that starts from an initial DNN state and evolves
the state following a trajectory dictated by the dynamics of the update step
equations until converging hopefully to an equilibrium state at which the DNN
state becomes stable. Thus, PBT can be applied on In this section, we
introduce an adaptation of property-based software testing (PBT) that
assembles several properties of training programs that should be satisfied by
the initial state (i.e., pre-training conditions), maintained for the
intermediate states of the on-training DNN (i.e., proper fitting conditions),
and validated for converged DNNs (i.e., post-training conditions). Besides, an
adaptation of input shrinking, as detailed in 4.6.5, can be applied to the DNN
state to determine at which level and in which component the property is
violated.

###  4.2. Origins and Types of DNN Training Properties

Over the last decade, we have witnessed a wide adoption of DL technology in
various industrial domains that represent the outgrowth of huge efforts and
dedications from the DL community into the knowledge transfer. Indeed, applied
DL researchers and experts vulgarize DL fundamentals and research advances in
sort of principles, techniques and tricks in order to get more practitioners
involved into applying DL technology for solving learning task problems. This
gave birth to popular applied DL textbook (Goodfellow et al., 2016), academic
lectures (MIT, 2020b, a), articles (Islam et al., 2020; Smith and Topin, 2016;
Hasanpour et al., 2018; Canziani et al., 2016), and industry courses (Google,
2020; Ng et al., 2020), as well as experts’ blogs (Ivanov, 2017; Shao, 2019;
Bansal, 2020; Deeplearning4j, 2019; Karpathy, 2020) and DL practitioners’
forum discussions (community, 2018; Karpathy, 2018a) on DNN troubleshooting
techniques and strategies.

Figure 5. Main Steps of Development Process for Verification Routines

Figure 5 summarizes the steps that we follow to construct automated
verification routines for the identified DL training pitfalls. From the
fundamental DL resources, we distill the main properties and design principles
for each neural network component for which issues have been detected. For
instance, we find the properties in relation to the initial random parameters,
the hidden and output activation functions, or the gradient-based optimization
routines. Basically, DL training programs are sensitive to the DL bugs that
violate some of the involved components’ properties. Nevertheless, the
detection of properties’ violations in a suspicious DL training program is
quite challenging. That’s why, we explore the heuristics and troubleshooting
strategies elaborated by DL experts, which rely mainly on plotting histograms
of model internals and curves of performance metrics, in order to spot
unexpected distributions and irregular curve shapes. For instance, Glorot and
Bengio (Glorot and Bengio, 2010) watched the activation distribution to detect
any possible layer saturation when they studied different random weight
initializations. Then, we analyze how these heuristics are able to
characterize the violations of properties. Generally, the heuristics specify
critical values for metrics that shed light on a faulty DL program’s state,
such as high ratio of null activations or high magnitude updates of
parameters. Besides, the heuristics can also describe erroneous training
behaviors that would manifest in the dynamics of the metrics, such as
unanticipated fluctuating or diverging loss. Even if these abnormal behaviors
could be captured and illustrated by experts through visualizations (e.g.,
histograms of activations, curve of losses over epochs, etc.), the
codification of automated verification routines, that detect the violation of
statistical learning properties and not-recommended instability for an on-
execution DL training program, is challenging. Indeed, the inherent iterative
nature and stochasticity of DNN training algorithms makes the regular
deterministic test assertions impractical because a single property-violating
state is not sufficient for asserting the occurrence of an issue. For
instance, the current state may trigger a dead layer (i.e., more than $50\%$
of ReLUs in a layer are null), but the next state following the updates can
avoid the problematic situation by reducing the inactive ReLUs. Hence, the
persistence of the property-violating state, catched by the heuristics, should
be taken into consideration to avoid overwhelming warnings and misleading
false alarms during the debugging sessions. Below, we explain the developed
guidelines to codify the DL experts’ heuristics into robust and dynamic
verification routines that are designed to assess persistent behavioral
issues.

####  4.2.1. Buffering and Statistical reductions

The parameters and internal computations in a deep neural network are volatile
multi-dimensional arrays. Thus, we define buffers to store the last
intermediary states, including the hidden activations, the predictions, the
losses, etc., in order to validate the heuristics on a set of recent states
instead of a single one. Then, we calculate different statistics on their
distributions over different axis of interest to reduce the dimensions and
create the most appropriate data views/metrics to handle in the codification
of the rule. For instance, the checks on the activation distribution would run
periodically on all the activations stored on the buffer, which are obtained
from the last training iterations (i.e., anticipated to be using different
parameters and batches of data). By default, we set up a buffer size that
equals $10$. Therefore, given the same example of dead layers, we reduce the
buffered activations, from $10$ values per neuron to a single one, using
$95^{th}$ percentile, which is a robust maximum estimation (highest value
under the top 5%). Then, our proposed check would flag the layers with more
than a half of dead neurons, which have returned $95\%$ of outputs below the
minimum threshold of $1e-5$ for the last $10$ training iterations.

####  4.2.2. Pessimistic boundaries

As TheDeepChecker consists of a debugging method, we set up pessimistic
thresholds to spot critical values and erroneous behaviors that are probably
caused by a DL bug. This conservative strategy can effectively reduce many
possible false alarms in relation to ineffective training traits that usually
manifest in earlier iterations or on complex learning problems. Nevertheless,
we keep the thresholds as user-configurable settings in order to adjust the
sensitivity of the verification routines on specific DL architectures
according to user interests.

####  4.2.3. Approximative and Behavioral assertions

The heuristic-guided DL program diagnostic requires the implementation of
approximative assertions including almost numerical equal assertions for
floating numbers and statistical significance tests to identify if the
obtained program state is outside the anticipated set of possible states by
the experts. For instance, there are no best initial parameters, however, DL
experts have shown the importance of sampling the random parameters at the
first iteration from a carefully-designed distribution, depending on the
neural network characteristics. Besides, crafting verification rules for
diverse abnormal DNN states such as stagnated loss, huge weights or vanished
gradients, etc., can be difficult because the involved metrics’ thresholds
would vary between models and problems. Thus, it is important to focus on
characterizing the erroneous behaviors: stagnation, diverging, or vanishing
instead of the resulting faulty state to which the buggy DNN training program
would converge. Indeed, we enable the detection of abnormal trends through the
assessment of their evolution over consecutive iterations, i.e., by
considering a window of steps, generally, window size would be in-between $3$
and $5$. In the following, we describe our proposed behavioral assertions for
the common erroneous behaviors resulting from the identified DL training
pitfalls:

##### Stagnation

It is the opposite of changing and moving quantity. Hence, we define a minimum
percentage difference by which the quantity of interest should change at each
step within the window; otherwise, we flag it as stagnated. More specifically,
the change direction is already known to compute a relative percentage of
increase or decrease, e.g., we expect that the loss keeps decreasing until
convergence to a minimum.

##### Diverging or vanishing

They represent unexpected huge increases and decreases in a quantity over
time. They can be simulated as exponential growth and decay,
$q_{t}=q_{0}\times r^{t}$ where, respectively, $r>1$ and $r<1$. Thus, we can
approximate the rate of change $r_{t}=\frac{q_{t}}{q_{t}-1}$ for a window of
recent steps, then, we consider that a quantity is diverging if the calculated
$r_{t}$ are higher than a low bound, or it is vanishing if all the $r_{t}$ are
lower than a high bound. In our constructed verifications, we used $2$ as
low_bound, meaning that the quantity should constantly double its value at
minimum to be considered a diverging quantity and inversely, we used
$\frac{1}{2}$ as high_bound. The component is flagged when it maintains this
unanticipated non-linear evolution for a predefined window of steps.

####  4.2.4. Consistency assessments

Several experts’ heuristics, that are incorporated into TheDeepChecker’s
verification routines, help recognize the unstable distributions of
parameters, activations and gradients, as well as unexpected optimization
updates and loss curves. However, the riskiness of these unstable learning
situations increases when the identified issue persists or becomes more severe
over the iterations. Hence, we smooth the verification logic by (1) computing
the rule’s metrics on the aggregation of previously-obtained states from the
buffer; (2) considering a forbearance period, which is a prefixed number of
steps to wait, despite the persistent failure of the verification rule, before
flagging the occurrence of the issue.

Finally, we select the debugging phase during which the codified verification
routine would run depending on their input DNN states. Indeed, we mainly
separate between the initial state verification, the validation of the under-
test DL program running on a batch of data, and the need for longer training
over larger datasets or comparison of multiple trained models. In the
following, we describe the chronological sequence of the debugging phases, as
illustrated in Figure 6 and we detail the heuristic and logic of all the
verification routines included in each phase of TheDeepChecker’s debugging
session.

Figure 6. Overview of TheDeepChecker Debugging Phases

###  4.3. Phase 1: Pre-training conditions

The first phase of our debugging process occurs before starting a training
session. It enables running static pre-checks of the input data and the
starting initial state of the on-training DNN. The benefit of these
preliminary verifications is to validate, from the start with a null training
cost, the quality of feeding data (i.e., input features, labels), the
correctness of essential implemented components (i.e., gradient, custom
operations), and the adequacy of the starting state (i.e., initial parameters,
first loss). In the following, we describe the pre-checks and their related
training pitfalls.

####  4.3.1. Data Distribution

DL practitioners often perform linear re-scaling of the input and output
features, in order to adjust their distribution into a common scale without
distorting differences between the ranges of original values. In fact, the two
most common data scaling techniques are: (1) standardization consists in
transforming the inputs into z-scores, which means that the transformed data
should have a zero mean and a unit standard deviation; and (2) normalization
consists in re-scaling each input feature using its maximum and minimum
elements, to have values within a predefined small range such as $[0,1]$ or
$[-1,1]$.

##### Scaled Data Verification

We extract the data, inputs and outputs, that are fed to the training program;
i.e., the final data that have been going through the preprocessing pipeline,
and then, verify that the data is scaled properly. In our verification
routine, we start by detecting the constant features that have a zero
variance. Then, we support checking whether the features are zero-centered
with unit standard deviation, or they belong to one of the two well-known
normalized range of values: $[0,1]$ or $[-1,1]$ (LeCun et al., 2012).
Moreover, this helps detect if the data accidentally includes undefined or
non-finite quantities (i.e., NaNs and Infs).

##### Unbalanced Labels Verification

We compute the Shannon equitability index (Shannon, 1948), which summarizes
the diversity of a population in which each member belongs to a unique group,
to estimate the balance between the frequencies of labels. On a data set of
$N$ instances, if we have $K$ labels of size $N_{k}$, we can Shannon
equitability index as follows,
$\frac{-\sum_{i=1}^{k}{\frac{N_{k}}{n}\log(\frac{N_{k}}{n})}}{\log(K)}$, and
it will be zero when there is one single label. Thus, it tends to zero when
the dataset is very unbalanced. In our verification routine, we use by default
the minimum threshold of $0.5$ to flag the labels data as unbalanced. This
will be reported to the user as warning and will affect the verification of
the initial bias.

####  4.3.2. Starting DNN state

Starting from different initial states, the optimization algorithm follows
different trajectories and can terminate at different equilibrium states.
Thus, a poor initial state adversely affects the optimization routines, and
consequently, the optimality of the equilibrium state.

##### Initial Weights Verification

First, we verify that there are substantial differences between the
parameter’s values by computing the variance of each parameter’s values and
checking if it is not equal to $0$. Next, one can make sure that, given the
chosen activation function, the distribution of initial random values are
sampled from a uniform or normal distribution with a careful tweak, i.e., by
calibrating attentively the variance because the distribution of the outputs
from a randomly initialized neuron has a variance that grows with the number
of inputs. The equality between the actual variance of each weight and its
recommended variance given the input size is verified using f-test
(Archdeacon, 1994). In the following, we describe the recommended variances
depending on the activation function of the corresponding layer.

  * -

LeCun (LeCun et al., 2012) proposes a heuristic that initializes each neuron’s
weight as either $\mathcal{N}(0,\sqrt{1/fan_{in}})$, i.e., normal distribution
with zero mean and $1/fan_{in}$ of variance or
$\mathcal{U}(-\sqrt{3/fan_{in}},+\sqrt{3/fan_{in}})$, i.e., uniform
distribution within $[-limit,limit]$ and $limit=\sqrt{3/fan_{in}}$, where
$fan_{in}$ is the number of inputs. This guarantees that all the initial
neurons’ weights have approximately the same output distribution, and its
empirical evaluation on sigmoid layer activation shows a significant
improvement on the rate of convergence.

  * -

Glorot et al. (Glorot and Bengio, 2010) recommend the following initialization
(especially when tanh is used as activation function) which consists of
neuron’s initialization following
either $\mathcal{N}(0,\sqrt{2/(fan_{in}+fan_{out})})$ or
$\mathcal{U}(-\sqrt{6/(fan_{in}+fan_{out})},+\sqrt{6/(fan_{in}+fan_{out})})$,
where $fan_{in}$, $fan_{out}$ are the number of inputs and outputs.

  * -

He et al. (He et al., 2015) also proposed an initialization specifically for
ReLU neurons. They suggest that the variance of neurons in the network should
be $2/fan_{in}$, which gives an initialization of either
$\mathcal{N}(0,\sqrt{2/fan_{in}})$ or
$\mathcal{U}(-\sqrt{6/fan_{in}},+\sqrt{6/fan_{in}})$. All of the above
initializations have been discovered empirically and proven to be optimal in
classic well-known CNNs like LeNet (LeCun et al., 2015) and modern
architectures such as VGG (Simonyan and Zisserman, 2014) and GoogleNet
(Szegedy et al., 2015).

##### Initial Biases Verification

As a baseline, we verify that the bias exists, and initially is set to $0$ in
case of well distributed labels. Nevertheless, we proceed by more advanced
check on the last bias initialization in case the pre-check on unbalanced
labels for classification was fired. Indeed, we make sure that the bias set
for the output layer reflects the bias already found in the distribution of
outcomes in the given ground truth data. In our implemented verification
routine, we consider the unbalanced classification problem, where it is
usually effective to set each bias unit $b_{i}$ to $\log(p_{i}/1-p_{i})$,
where $p_{i}$ is the proportion of training instances of the label
corresponding to the bias $b_{i}$ of unit $i$ (Hinton, 2012). Concerning the
regression problem, if the coefficient of variation w.r.t each output $j$
(i.e., the ratio of the standard deviation to the mean) is low (e.g., our
default threshold is $0.1\%$), we verify that its corresponding $b_{j}$ is set
to $m_{j}$, the mean value of the supervised target $j$. This eases the
optimization by transforming the regression problem into predicting the
deviation against the baseline (i.e., the mean value).

##### Cold Start Loss Verification

An unexpected loss at the iteration $0$ with an untrained model (cold start),
can indicate numerous issues including faulty loss function, ineffective loss
reduction strategy, as well as buggy parameters’ initializers (Karpathy,
2018b). Indeed, we compute the loss at cold start with an increasing size of
batches in order to verify that the obtained losses are not proportionally
increasing. This validates that the loss estimation is an average-based
expectation and it is not a sum-based reduction. In our verification routine,
we duplicate a random batch of data by doubling its size ($\times 2$), then,
we check if their corresponding losses at cold start are doubling ($\times 2$)
as well. Next, we verify that the optimizer starts well at the expected
initial loss (i.e., the one estimated at the first run with random internal
parameters). It is always possible to derive approximately the correct initial
loss for a given DNN program configuration. For instance, the cross-entropy
loss for a balanced classification problem should start with uniformly
distributed probabilities of $p(label)=1/L$ and initial value of
$loss=-\log(1/L)$, where $L$ is the number of target labels.

##### Tensor-Based Operation Verification

A careless developer can introduce mistakes when transforming math formulas or
pseudo-code from white papers to tensor-based operations using basic DL
libraries calculation APIs. As the DNN tensor-based computations include
intensive broadcasting and reduction operations in order to perform
individually calculation over all the instances/neurons at each level, then,
reduce the calculations to define aggregative scores towards summarize all
into a scalar cost that represent how well the DNN performs on the data. We
found that common implementation mistakes miss or add unnecessary dependencies
to network components (instances, neurons, scores,..etc). Thus, it is
important to test that a written operation depends only on its related
components. For example, an activation function should be applied separately
on all the neurons, which means the activation output $i$ should depend only
on the neuron input $i$, or a distance calculation between prediction and
actual values should contain independent components, where the distance value
$i$ depends on only the prediction $i$ and the ground truth label $i$ as well.
To validate the correctness of dependencies of all the math operations for a
computed quantity within the neural network, we use the gradients flowing in
the network to debug the dependencies between each operation’s components
given the fact that the gradient of a function w.r.t an independent component
is always zero (Karpathy, 2020). For instance, let’s consider a newly-
implemented loss function, we can extract only the loss obtained for the
outputs of a data point $i$, and then perform a full backward pass to the
input data in order to make sure that only the gradient w.r.t the $i$-th input
data is not null. A violation of this condition signals that overlapping
dependencies exist, which means that the on-watch average loss for DNN
performance is wrong and misleading.

##### Computed Gradient Verification

The backbone of backpropagation implementation lies in the computation of
gradients with respect to different DNN operations, including linear weighted
sum and non-linear activations. Whenever DL developers add hand-crafted math
operations and gradient estimators, we perform a numerical gradient checking
that consists of comparing between the analytic and numerical calculated
gradients, respectively, the gradient produced by the analytic formula and the
centered finite difference approximation,
$\frac{f\left(x+h\right)-f\left(x-h\right)}{2h}$. Both gradients should be
equivalent, approximately equal, for the same data points. The following steps
are recommended by (Karpathy, 2018b; Goodfellow et al., 2016) to improve the
effectiveness of this gradient checking process (and the detection of faulty
gradients).

Relative error comparison: The difference between numerical gradient
$f_{n}^{\prime}$ and analytic gradient $f_{a}^{\prime}$ represents the
absolute error that should not be above a predefined threshold. However, it is
hard to fix a common threshold of absolute error for DNN because its internal
computations are usually composed of multiple functions; so the errors build
up through backpropagation. Thus, it is preferred to use a relative error,
$\frac{|f_{a}^{\prime}-f_{n}^{\prime}|}{max(|f_{a}^{\prime}|,|f_{n}^{\prime}|)}$,
that might be acceptable below $1e^{-2}$.

Sampled data instances: Sampling a few data instances for numerical gradient
checking reduces the risk of crossing kinks, which are non-differentiable
areas of the loss landscape. For instance, ReLU has zero gradient at the
origin; but the numerical gradient can cross over the kink and produce a value
different from zero. Besides, DNN’s parameters are large with thousands of
dimensions; so the computation error could be on a random subset of each
gradient dimension. Therefore, a random sampling among both data points and
dimensions makes the finite-difference approximations less error-prone and
faster in practice.

No regularization: The standard regularization can render large errors,
misleading the numerical gradient checking when the penalty term added
overwhelms the original loss (i.e., the gradient is mostly related to penalty
cost). Moreover, advanced regularization techniques such as dropout induce
non-determinism in the DNN internal computations, which enhances the error-
proneness of numerical gradient checks.

Prior burn-in training: A short burn-in training during which the parameters
take better and more representative values than randomly initialized ones is
recommended. It is also recommended to avoid the gradient checking at cold
start since it could introduce pathological edge cases, masking a buggy
implemented gradient.

####  4.3.3. Fitness of a single batch of data

Given a tiny sample of data, the target problem becomes easy to solve and the
training algorithm should be able to converge to a DNN that fits the data
without any issue, as every well-designed DNN should be able to fit a small
dataset. This is a main pre-check for DNN training routines because it is a
necessary condition, where its non-satisfaction indicates a misconfiguration
or a software bug.

##### Input Dependency Verification:

We confirm that training programs on zeroed batches of data perform worse than
those on real samples of data. This check was initially proposed by Karpathy
(Karpathy, 2020), as a verification that the model outperforms an input-
independent baseline. For debugging, this improvement over the input-
independent baseline shows that the training program is successfully
leveraging the input to optimize the DNN parameters.

##### Overfit Verification:

We verify that the optimization mechanism is working well on a controlled
sample of data with reduced size (i.e., a few data points for each class)
(Goodfellow et al., 2016; Karpathy, 2018b). The acceptance criteria is that
the DNN achieves $100\%$ accuracy or near-to-zero absolute errors (AEs) on
continuous outputs (by default, we consider AE in the order of $10^{-3}$). A
failure to achieve this performance would signal the presence of issues
regarding the DNN optimization routines.

##### Regularization Verification:

As above-mentioned, the controlled experiment of fitting a single batch would
lead to overfit the provided few data points in normal situations, however,
the loss should be greater than zero (Goodfellow et al., 2016) when there is
quite regularization. This check can be improved by watching furthermore the
smoothness of the loss curve to spot a lack of noise in the optimization, and
consequently, it reinforces doubts about the absence of active regulation. In
our verification routine, we set up $1e-5$ and $0.95$ as default thresholds
for loss and smoothness ratio 4.4.1 to recognize suspicious loss curves that
are smoothly decreasing towards a very low loss. This shows the model’s
propensity to overfit quickly the training data caused by a lack of
regularization, which often implies high risks of capturing useless residual
variations in the given features.

Regarding pre-check of a single batch of data fitness, we concentrate on the
functioning of the training algorithm, and precisely, its ability to converge
(i.e., reaching an optimal equilibrium state) given a reduced size problem.
However, the DNN training may still contain inefficiencies that did not
prevent it from solving a small problem size but would affect its performance
when it trains on larger problem size. Moreover, the failure at these batch
fitness prechecks do not provide indications about the possible root causes
behind this incapacity to successfully pass them. Therefore, the next phase in
our debugging process aims to guarantee the “proper functioning” of the
training algorithm given a reduced size problem. What we mean by “proper
functioning” is the ability to converge with a valid accomplished trajectory
(i.e., passing through valid intermediary DNN states).

###  4.4. Phase 2: Proper fitting a single batch of data

At this phase, a monitored training is launched on a representative sample of
data (i.e., a single batch of data resulting from a stratified sampling). The
monitoring routines serve to detect early and precisely the potential issues
with diverse automated verifications that watch periodically for the DNN
components’ misbehaviors to spot and report properties’ violations. In case of
a DNN with major issues, i.e., already failed the precheck on single batch
fitness, this phase allows to steer the user towards problematic components
and provide meaningful messages on the violated properties that restrict
further the potential root causes. Otherwise, DL practitioners can still
leverage this debugging phase for pre-examined DNNs to spot inefficiencies of
their design choices that cause training instability in regards to learning
updates or activation distributions, which might prevent the DNN from reaching
optimal steady states and lead to a wastage of time and resources on
unnecessary long training sessions with the full datasets.

Figure 7. Illustration of loss minimization issues

####  4.4.1. Abnormal Loss Curvature

A loss curve is a plot of model loss value over time in terms of epochs or
iterations. The shape and dynamics of a loss curve are useful to diagnose the
behavior of the optimizer against the target learning problem. Figure 7 shows
the abnormal loss curves, detailed-below, which indicate different pathologies
of statistical learning from data. The anomalous loss evolution (Karpathy,
2018b) can be detected using continuously updated metrics that are cheap to
compute and which can reveal anomalies effectively.

##### Non- or Slow-Decreasing loss

A flat or low slope down curve shows that the loss is either non-decreasing or
decreasing very slowly which means that the model is not able to learn at all
or has a low learning capacity. This could be due to an inadequate loss
function or a low learning rate. As introduced in 4.2.3 for the stagnation
test, we verify that the loss is decreasing at an acceptable decay rate for a
window of steps, e.g., we set up $5$ steps by default. For the loss decreasing
verification, we proceed by watching that the percentage difference between
consecutive losses’ values, $loss\\_decay\\_ratio=\frac{previous\\_loss-
current\\_loss}{previous\\_loss}$, is greater or equal to a predefined minimum
percentage difference (by default, we set up $5\%$) by which the loss should
decrease, $loss\\_decay\\_ratio>prefixed\\_min\\_loss$.

##### Diverging loss

A curve with a high slope indicates that the loss is diverging with wildly
increasing values which means that the optimization problem is turned into a
loss maximization instead of minimization. This could be the result of a high
learning rate or a buggy gradient. Therefore, we verify that the loss has no
increasing tendency instead of decreasing one. This verification can be done
automatically by updating a reference minimum loss ($lowest\\_loss$), and
then, watching that the absolute ratio of loss,
$abs\\_loss\\_ratio=\frac{current\\_loss}{lowest\\_loss}$ (i.e., we call it
absolute ratio because it is computed w.r.t the lowest obtained loss) is not
dramatically diverging, as described in 4.2.3, during the training.

##### Highly-Fluctuating loss

A noisy curve with random fluctuations demonstrates that the loss is not
converging to a line of stability, hence, the optimization is facing
difficulties preventing it from converging normally. Potential reasons behind
this issue could be: strong regularization that gives rise to noisy loss
estimation, or high learning rate that produces large updates keeping the
optimizer jumping over the local minimum without converging. Hence, we compute
the smoothness ratio of the loss curve as follows:

$smoothness\\_ratio=\frac{N\\_samples-N\\_direction\\_changes}{N\\_samples}$

Where $N\\_samples$, $N\\_direction\\_changes$ denote, respectively, the
sampled iterations count and the number of alternations in the loss evolving
direction. Then, we check periodically that the smoothness ratio is not lower
than a predefined threshold, otherwise, it indicates a high amount of
fluctuations (e.g., we fix $0.5$ as a default threshold, which means that more
than $50\%$ of consecutive sampled losses are having altered directions).

####  4.4.2. Performance Metrics Correlation

DL practitioners should select or implement the right loss measure (i.e., mean
squared error, cross-entropy, etc.) that will be set as objective function for
parameters optimization, while they keep watching a target performance metric
(i.e., R-squared, Accuracy, etc.). For this verification routine, we compute,
continuously, the absolute value of correlation coefficient between the
optimized loss and the target performance measurements between training steps.
The latter describes the magnitude of the relationship between two variables
within the interval of $[0,1]$ and should not become lower than a predefined
threshold (e.g., our verification routine reports an absolute correlation
coefficient of less than $0.5$). This provides an indicative metric of how
representative the optimized loss is with respect to the target objective and
vice-versa. Thus, a poor choice of loss function would have high chances to be
uncorrelated with the performance metric, similarly, a buggy performance
calculation would yield incorrect values with low correlation with the loss
evolution.

####  4.4.3. Unstable Gradient

We propose to detect unstable gradient issues by examining, continuously, the
evolution of estimated gradient’s values with respect to each DNN’s layer.
More specifically, we check the growth and decay rate of the absolute average
of the layer’s gradients, estimated for the last iterations, to detect if the
latter is following an unstable evolution trend, i.e., it is exploding or
diminishing, as described in 4.2.3.

####  4.4.4. Magnitude of Regularization Penalty Cost

In addition, there is an issue with the addition of a high amount of
regularization to the computed loss, whether by using a high $lambda$ value or
a poorly-designed penalty regularization cost. Indeed, Park et al. (Park et
al., 2018) highlight that the learning suddenly fails when the magnitude of
gradients from $loss(W,b,D)$ decreases faster than that from $\Omega(W)$; so
the penalty term gradient overwhelm the loss data gradient. In such
problematic situations, the weights’ updates become mainly related to the
regularization term, which causes the failure of the model to learn.
One should ensure that the regularization is not too strong. The
regularization term gradient should not dominate and suppress the loss
gradients w.r.t the weights.To ensure this, we recommend watching
continuously, the proportion of the magnitude of the penalty terms’ gradients
w.r.t the magnitude of loss data in order to validate that it is not
diverging, as described in 4.2.3.

####  4.4.5. Parameters States and Dynamics

The main DNN’s on-training parameters, weights and biases, should be optimized
towards better solving the learning problem over the training sessions. In the
following, we propose different verification routines on the parameters
current states and update dynamics that could indicate convergence issues and
non-optimality of the on-going estimation.

##### On-Training Parameters Verification

A preliminary sanity check for neural network parameters would be verifying
their changing estimations over the training iterations. We make sure that all
defined trainable layers have their parameters updated. Indeed, a non-zero
difference between the values of trainable parameters before and after the
execution of a few training passes (i.e., optimization updates) confirms that
the dependencies between the layers are correctly set up and that all the
trainable parameters are getting optimized.

##### Dead and Over-Negative Weights Verification

A layer’s weights are considered dead or over-negative, when respectively, the
ratio of zeros or negative values in the tensor elements is very high (Evci,
2018). These two states of weights are likely to be problematic for the
learning dynamics. Indeed, given a common DNN setup (e.g., normalized inputs
within $[0,1]$ and a variant of ReLU as hidden layers activations), null or
negative learned weights within hidden layers represent connections to
intermediate features that do not contain any relevant information for the
target task. Thus, if a layer’s weights are mostly full of null or negative
weights, their corresponding activation layers are likely to stagnate on a
non-optimal flat region and consequently, the DNN would start facing dead
layers (i.e., ReLUs mostly outputting the value zero) and frozen layers (i.e.,
no updates of the weights). In our verification routine, we flag any tensor of
layer’s weight ($W$), having either the ratio of very low values (i.e., lower
than $1e-5$) or negative values is higher than a predefined threshold (i.e.,
$95\%$ is used by default), as respectively, dead or over-negative weights.

##### Stable Parameters Update Verification

Deep neural networks introduce challenges regarding learning stability
compared to shallow networks. In the training pitfalls section 3, we discussed
some practices such as tuning the learning rate and adding activation or
weight normalization to balance out the learning speeds for the hidden layers.
Thus, it is important to make sure that the parameters’ updates are stable.
Hinton (Hinton, 2012) and Bottou (Bottou, 2015) proposed the following
heuristic, the magnitude of parameter updates over batches should represent,
respectively, $0.1\%$ or $1\%$, of the magnitude of the parameter itself, not
$50\%$ or $0.001\%$. Therefore, we propose a verification routine to detect
unstable learning parameters by comparing the magnitude of parameters’
gradients to the magnitude of the parameters themselves. More specifically,
following the recommendation to keep the parameter update ratio around $0.01$
or $0.001$ (i.e., $-2$ or $-3$ on base 10 logarithm), we compute the ratio of
absolute average magnitudes of these values and verify that this ratio doesn’t
diverge significantly from the following predefined thresholds:

$-4<\log_{10}\left(\overline{|W^{(i+1)}-W^{(i)}|}/\overline{|W^{(i)}|}\right)<-1$

The proposed verification mechanism reports irrelevant layers (i.e., where
updates are unstable) and frozen layers (i.e., where updates are stalled) to
the user.

##### Parameters Diverging Verification

Worse than unstable learning, weights and biases risk divergence, and may go
towards $+/-\infty$. For instance, high values of initial weights or learning
rate with a lack of–or–insufficient regularization provokes highly-increasing
weights’ updates, leading to bigger and bigger values, until reaching $\infty$
(this is caused by overflow rounding precision). In addition to that, biases
can also become huge in certain situations where features could not explain
enough the predicted outcome or might not be useful in differentiating between
the classes. Therefore, we automate a verification routine that watches
continuously the absolute averages of parameters are not diverging, as
described in 4.2.3.

####  4.4.6. Activations Distribution

##### Out-of-Range Activation Verification

Given a newly-implemented activation function, we include a baseline
verification routine to watch that the produced activations are within the
expected range of values. This would be useful to find computation mistakes or
misconceptions causing out-of-range outputs.

##### Validation of Output Activation Domain

The last layer’s activation represents the outputs of the neural network,
which should produce valid outcomes while covering the whole distribution of
the possible ground truth labels. We implement a verification routine to check
that the outputs of classifiers are probabilities, i.e., positive values
within $[0,1]$ and sum-to-one in case of multidimensional output. For
regression, we empirically verify that the predicted outputs over the
iterations were able to satisfy necessary conditions derived from ground truth
boundaries, i.e., non-zero variance, can be negative, can exceed $1.0$. Under
these conditions, the common identified faults of using the wrong activation
function can be detected.

##### Saturated Bounded Activation Detection

To detect saturation issues in DNNs, we compute single-valued saturation
measure $\rho_{B}$ proposed by Rakitianskaia and Engelbrecht (Rakitianskaia
and Engelbrecht, 2015) if the hidden activations are bounded functions such as
sigmoid or tanh. This measure is computed using the outputs of an activation
function and is applicable to all bounded activation functions. It is
independent of the activation function output range and allows a direct
statistical measuring of the degree of saturation between NNs. $\rho_{B}$ is
bounded and easy to interpret: it tends to $1$ as the degree of saturation
increases and tends to zero otherwise. It contains a single tunable parameter,
the number of bins $B$ that converges for $B\geq 10$, i.e., it means splitting
the interval of activation outputs into $B$ equal sub-intervals. Thus, $B=10$
can be used without any further tuning. Given a bounded activation function
$g$, $\rho_{B}$ is computed as the weighted mean presented in Equation 15.

(15)
$\rho_{B}=\frac{\sum^{B}_{b=1}|\bar{g}^{\prime}_{b}|N_{B}}{\sum^{B}_{b=1}N_{B}}$

Where, $B$ is the total number of bins, $\bar{g}^{\prime}_{b}$ is the scaled
average of output values in the bin $b$ within the range $[-1,1]$, $N_{b}$ is
the number of outputs in bin $b$. Indeed, this weighted mean formula turns
into a simple arithmetic mean when all weights are equal. Thus, if
$\bar{g}^{\prime}_{b}$ is uniformly distributed in $[-1,1]$, the value of
$\rho_{B}$ will be close to $0.5$, since absolute activation values are
considered, thus all $\bar{g}^{\prime}_{b}$ values are squashed to the $[0,1]$
interval. For a normal distribution of $\bar{g}^{\prime}_{b}$, the value of
$\rho_{B}$ will be smaller than $0.5$. The higher the asymptotic frequencies
of $\bar{g}^{\prime}_{b}$, the closer $\rho_{B}$ will be to $1$.
This verification routine can be automated by storing for each neuron its last
$O$ outputs’ values in a buffer of a limited size. Then, it proceeds by
computing its $\rho_{B}$ metric based on those recent outputs. If the neuron
corresponding value tends to be $1$ (e.g., a threshold of $0.95$ is used in
practice to spot this tendency), the neuron can be considered as saturated.
After checking all neurons for saturation, we compute the ratio of saturated
neurons per layer to alert the DL developers about layers with saturation
ratios that surpass a predefined threshold (e.g., we fix $50\%$ by default).

##### Dead ReLU Activation Detection

By definition, a given neuron is considered to be dead if it always returns
zero (Lu et al., 2019). Hence, we detect practically dead ReLUs by reducing
the last outputs for each neuron stored in the limited size buffer into a
single $95^{th}$ percentile, which is more robust than the maximum reduction
against outliers (e.g., a non-zero stored values from earlier training
iterations). Thus, we mark all the neurons with a $95^{th}$ percentile less
than a predefined threshold (i.e., by default, we set up $1e-5$). Next, we
proceed by calculating the ratio of the marked dead neurons per layer and we
flag the layers with a number of dead neurons higher than a predefined
threshold (e.g., we fix a default threshold of $50\%$).

##### Unstable Activation Detection

Although this unstable activation issue is more generic than dead or
saturation phenomena, DL experts usually watch the histograms of sampled
activations from each layer while expecting to have normally-distributed
values with unit standard deviation, e.g., a value within $[0.5,2]$ has been
shown to be an acceptable variance of activations (Jing Zhi Loh and Azeem,
2020). Thus, we base on this expert’s heuristic to statistically validate that
the sampled activations of each layer over the last iterations is having a
well-calibrated variance scale. Concretely, the test would pass the actual
standard deviation ($\sigma_{act}$) belongs to the range of $[0.5,2]$;
otherwise, we perform an f-test (Archdeacon, 1994) to compare $\sigma_{act}$
with either the low-bound $0.5$ if $\sigma_{act}<0.5$ or $2.0$ if
$\sigma_{act}>2.0$.

DL practitioners can perform a closed feedback loop using this inexpensive and
rapid debugging on a single batch of data until fixing all the covered coding
bugs and improving the settings in a way that enhances the chances of
converging to a more optimal model. Thus, a successful pass at this phase
increases the confidence that the training program is devoid of common DNN
pathologies such as: vanishing gradients, saturation of activation functions,
and inappropriate learning speed. Nevertheless, other components of the
training program have not been tested yet. For instance, a training program
may pass all the single batch debugging checks, while the data loader can
inject too much noise or mismatch features and labels, which yield corrupted
batches of data, and consequently, the resulting DNN does not solve the target
problem.

###  4.5. Phase 3: Post-fitting conditions

Once the fitting of a single batch step is successfully passed, several post-
fitting conditions should be satisfied to guarantee the correctness of the
data loader, the data augmentation module, and the advanced regularization
techniques that require additional computations during the inference. The
following debugging phase validates the behavior of the DNN training program
during regular training sessions, i.e., we use the available training and
validation data.

####  4.5.1. Distribution-Shifting Augmentation Verification

We propose a post-check that verifies the validity of the augmentation
methods, applying data transformations on the generated batches to enhance
diversity and improve the generalizability by smoothing the loss landscape and
forcing the model to capture the invariants useful for the target task. A
valid augmenter should not introduce a shift in the data distribution that
makes the model perform worse on the original dataset. For instance,
overwhelming noise injection leads to produce meaningless inputs; so both of
ratio and scale of the injected noise should be carefully picked to hold the
augmented data distribution close to the original one.
To detect this poor design of data transformations, we debug the data
augmenter module by comparing the performance and the activation patterns of
the DNN trained with-and-without augmentation on a sample of data from the
validation set. Concerning the measurement of dissimilarity between the
activation patterns, we use Centered Kernel Alignment (CKA) (Kornblith et al.,
2019), which is an optimized and powerful representational similarity measure,
allowing the assessment of the differences and the correspondences between
patterns learned by different DNNs or same DNN with different data. Indeed,
given two matrices $X$,$Y$, where $X\in\mathbb{R}^{m\times n_{1}}$ is a matrix
of activations of $n_{1}$ neurons for $m$ examples and
$Y\in\mathbb{R}^{m\times n_{2}}$ is a matrix of activations of $n_{2}$ neurons
for the same $m$ examples, a DNN representational similarity index $s(X,Y)$
estimates the similarity between the representations learned in both matrices
of activations. Thus, the CKA’s empirical validation (Kornblith et al., 2019)
shows its ability to compare representations within and across DNNs, in order
to assess the effect of changing variation factors on the DNN training. This
similarity metric is robust and it does not affect by the stochasticity of the
optimizer or the random initializations, which makes it suitable for our
verification on the resulting activation patterns instability when the
augmented data is noisy and shifted w.r.t the training data distribution.
Therefore, the DNN training program would fail the test if there is a
degradation of the performance and a substantial difference in the activation
patterns between the two trained DNNs. In our implementation, we check for an
increase in the loss ($loss_{augm\\_data}>loss_{orig_{d}ata}$) and a decrease
of activation pattern similarity (e.g., we set up a threshold of a minimum CKA
index equals to $0.8$, which means a decrease of $20\%$). Indeed, any
difference in the activation pattern could be acceptable and might be
considered as an improvement in the detected patterns in case the performance
is enhanced, however, having both of the activations pattern divergence and
the performance degradation indicate strongly that the augmented data induce a
distribution shift.

####  4.5.2. Corrupted Labels

Given a corrupted data shuffler, the paired data (features $X$, labels $y$)
are mismatched, where the row label $y_{i}$ does not correspond to the row
features $X_{i}$. Since the shuffling is executed after each epoch (i.e., full
pass over the data), a corrupted data loader will generate a new distribution
of supervised training data every time it is called. Thus, the training loss
curve would be subject to intermittent spikes because the neural network