# Why Deep Learning’s Performance Data Are Misleading

Juyang Weng<EMAIL_ADDRESS>0003-1383-3872 Brain-Mind Institute and GENISAMA4460
Alderwood Dr.OkemosMIUSA48864

(2023)

###### Abstract.

This is a theoretical paper, as a companion paper of the keynote talk at the
same conference AIEE 2023. In contrast to conscious learning, many projects in
AI have employed so-called “deep learning” many of which seemed to give
impressive performance. This paper explains that such performance data are
deceptively inflated due to two misconducts: “data deletion” and “test on
training set”. This paper clarifies “data deletion” and “test on training set”
in deep learning and why they are misconducts. A simple classification method
is defined, called Nearest Neighbor With Threshold (NNWT). A theorem is
established that the NNWT method reaches a zero error on any validation set
and any test set using the two misconducts, as long as the test set is in the
possession of the author and both the amount of storage space and the time of
training are finite but unbounded like with many deep learning methods.
However, many deep learning methods, like the NNWT method, are all not
generalizable since they have never been tested by a true test set. Why? The
so-called “test set” was used in the Post-Selection step of the training
stage. The evidence that misconducts actually took place in many deep learning
projects is beyond the scope of this paper.

††copyright: acmlicensed††journalyear: 2023††conference: 2023 The 4th
International Conference on Artificial Intelligence in Electronics Engineering
(AIEE); January 06–08, 2023; Haikou, China††booktitle: 2023 The 4th
International Conference on Artificial Intelligence in Electronics Engineering
(AIEE) (AIEE 2023), January 06–08, 2023, Haikou, China††price: 15.00††isbn:
978-1-4503-9951-7/23/01††doi: 10.1145/3586185.3586188††ccs: Computing
methodologies††ccs: Machine Learning††ccs: Learning Paradigms††ccs:
Unsupervised Learning

##  1\. Introduction

Since 2012, AI has attracted much attention from public and media. A large
number of projects in AI have published (LeCun et al., 2015; Bellemare et al.,
2020; et al., 2015; Silver et al., 2016; Graves et al., 2016; Silver et al.,
2016, 2017, 2018; Moravcik et al., 2017; Senior et al., 2020; McKinney et al.,
2020; Schrittwieser et al., 2020; Senior et al., 2020; Bellemare et al.,
2020), including AlphaGo, AlphaGoZero, AlphaZero, AlphaFold, and IBM Debater.
. If the authors of these projects understand the principles in this report,
they could benefit much for reducing the time and manpower to reach their
target systems as well as improving the generalization powers of their target
systems.

So-called “Deep Learning” has two steps in the training stage (Weng, 2021b).
The training stage has two steps: First, fit multiple systems each starting
with random weights (and try many hyper-parameters) using a training set.
Second, Post-Select the luckiest fit system based on validation set and test
set. Therefore, so-called “Deep Learning” is without any test stage.

This paper, based on the analysis of Post-Selections (Weng, 2021b), raises two
flaws that seem to widely exist in machine learning projects: (1) data
deletion and (2) test on training data. The latter is applicable when test
data are in the possession of the authors. All authors of published papers
that report authors’ own tests are in the possession of the test sets.

An open-competition is different, such as Deep Blue versus Garry Kasparov Feb.
10, 1996 - May 11, 1997, AlphaGo versus Lee Sedol March 9, 2016 - March 15,
2026, and AlphaGo versus Ke Jie May 23, 2007 - May 27, 2017, because the test
data arrives on the fly. This paper defines Post-Selections Using Test Set
(PSUTS) On The Fly (OTF) conducted by humans behind the scene. The author does
not claim that PSUTS OTF indeed took place during any of these three events
(although Kasparov did). The PSUTS OTF mode is for the academic community to
be aware of, to be alert about, and to investigate in the future.

In the remainder of the paper, we will discuss four learning conditions in
Sec. 2 from which we can see that we cannot just look at superficial “errors”
without limiting resources. Sec. 3 discusses four types of mappings for a
learner, which gives spaces on which we can discuss errors. Post-Selections
are discussed in Sec 4. Section 5 provides concluding remarks.

##  2\. The Four Learning Conditions

First, let us consider four learning conditions that any fair comparisons of
AI methods should take into account.

Many AI methods were evaluated without considering how much computational
recourses are necessary for the development of a reported system. Thus,
comparisons about the performance of the system have been tilted toward
competitions about how much resources a group has at its disposal, regardless
how many networks have been trained and discarded, and how much time the
training takes.

Here we explicitly define the Four Learning Conditions for development of an
AI system:

###### Definition 2.1 (The Four Learning Conditions).

The Four Learning Conditions for developing an AI system are: (1) A body
including sensors and effectors, (2) a set of restrictions of learning
framework, including whether task-specific or task-nonspecific, batch learning
or incremental learning; (3) a training experience and (4) a limited amount of
computational resources including the number of hidden neurons.

For example, the ImageNet competition (Russakovsky et al., 2015) did not seem
to explicitly restrict conditions (2), (3) and (4). The given images and class
labels correspond to restriction in condition (1). The AIML Contests (Weng et
al., 2018) considered all the four in performance evaluation.

Weng 2021 (Weng, 2021b) discussed the conditions (2) to (4) without condition
(1). It further discussed why any Big Data set violates what is called the
sensorimotor recurrence principle, namely, any learning process that uses a
static Big Data set is physically flawed.

##  3\. Four Different Mappings

Traditionally, a neural network is meant to establish a mapping $f$ from the
space of input $X$ to the space of class labels $L$,

(1) $f:X\mapsto L$

(Funahashi, 1989; Poggio and Girosi, 1990). $X$ may contain a few time frames.

Many temporal problems, such as video analysis problems, speech recognition
problems, and computer game-play problems, can include context labels in the
input space, so as to learn

(2) $f:X\times L\mapsto L.$

where $\times$ denotes the Cartesian product of sets.

A developmental approach deals with space and time in a unified fashion using
a neural network such as Developmental Networks (DNs) (Weng, 2011) whose
experimental embodiments range from WWN-1 to WWN-9. The DNs went beyond vision
problems to attack general AI problems including vision, audition, and natural
language acquisition as emergent Turing machines (Weng, 2015). DNs overcame
the limitations of the framewise mapping in Eq. (2) by dealing with lifetime
mapping without using any symbolic labels:

(3) $f:X(t-1)\times Z(t-1)\mapsto Z(t),t=1,2,...$

where $X(t)$ and $Z(t)$ are the sensory input space and motor input-output
space, respectively.

Note that $Z(t-1)$ here is extremely important since it corresponds to the
state of a Turing machine. Namely, all the errors occurred during any time of
each life is recorded and taken into account in the performance evaluation.
Different from the space mapping in Eq. (1) and very important, the space
$Z(t)$ is the directly teachable space for the learning system, inspired by
brains (Super, 1976; Thoroughman and Taylor, 2005; Rizzotti et al., 1987;
Moore et al., 2003; Thoroughman and Taylor, 2005; Iverson, 2010). This new
formulation is meant to model not only brain’s spatial processing (Weng and
Luciw, 2012) and temporal processing (Weng et al., 2013), but also Autonomous
Programming for General Purposes (APFPG) (Weng et al., 2020; Weng, 2020a).
Based on the APFGP capability, the AI field seems to have a powerful yet
general-purpose framework towards conscious machines (Weng, 2020b).

We need to consider two factors: (A) Space: Because $X$ and $Z$ are vector
spaces of sensory images and muscle neurons, we need internal neuronal feature
space $Y$ to deal with sub-vectors in $X$ and $Y$ and their spatial
hierarchical features. (B) Time: Furthermore, considering symbolic Markov
models, we also need further to model how $Y$-to-$Y$ connections enable
something similar to higher and dynamic order of time in Markov models. With
the two considerations (A) Space and (B) Time, the above lifetime mapping in
Eq. (3) is extended to:

(4) $f:X(t-1)\times Y(t-1)\times Z(t-1)\mapsto Y(t)\times Z(t),t=1,2,...$

in DN-2. It is worth noting that the $Y$ space is inside a closed “skull” so
it cannot be directly supervised. $Z(t-1)$ here is extremely important since
it corresponds to the state of an emergent Turing machine.

Asim Roy (Roy, 2008) argued that there are some parts of the brain that
control other parts. Here the area $Z$ could be treated as a regulatory
“controller” that regulates hidden neurons in the $Y$, e.g., as sensorimotor
rules. In neuroscience, there have been many published models that handcraft
areas as top-down regulators. In the DN model below, such areas must be
automatically generated and refined ML-optimally inside the closed skull
across lifetime.

In terms of performance evaluation, all the errors occurred during any time in
Eq. (4) of each life is recorded and taken into account in the performance
evaluation. This is in sharp contrast with Post-Selection.

##  4\. Post-Selections

Before we discuss Post-Selections, we need to discuss three types of errors.

###  4.1. Fitting, Validation and Test Errors

Given an available data set $D$, $D$ is divided by a partition into three
mutually disjoint sets, a fitting set $F$, a validation set $V$, and a test
set $T$ so that

(5) $D=F\cup V\cup T.$

Two sets are disjoint if they do not share any elements. The validation set is
possessed by the trainer, the test set should not be possessed by the trainer
since the test should be conducted by an independent agency. Otherwise, $V$
and $T$ become equivalent.

Given any hyper-parameter vector $\mathbf{a}_{i}$ (e.g., including receptive
fields of neurons), it is unlikely that a single network initialized by a set
of random weight vectors can result in an acceptable error rate on the fitting
set, called fitting error, that the error-backprop training intends to
minimize locally. That is how the multiple sets of random weight hyper-
parameter vectors come in. For $k$ hyper-parameter vectors $\mathbf{a}_{i}$,
$i=1,2,...k$ and $n$ sets of random initial weight vectors $\mathbf{w}_{j}$,
the error back-prop training results in $kn$ networks

$\\{N(\mathbf{a}_{i},\mathbf{w}_{j})\;|\;i=1,2,...,k,j=1,2,...,n\\}.$

Error-backprop locally and numerically minimizes the fitting error $f_{i,j}$
on the fitting set $F$.

Figure 1. A 2D-terrain illustration for the global minima problems in a high-
dimensional terrain (e.g., 200B-dimensional space of hyper parameters and
weights). TM: Turing machine.

Fig. 1 gives a 2D illustration for the limitations of Post-Selection as well
as the difference between the sensor-only mapping in Eq. (1) and the
sensorimotor mapping in Eq. (3). Suppose $X$ represents the space of sensory
input and $Z$ represents the space of motor input, but in general should be
any initial pair $(\mathbf{a}_{i},\mathbf{w}_{j})$. Each location on the 2D
plane of Eq. (1) corresponds to the initial random weights of a neural network
(e.g., 200B-dimensional). Of course, each network has many neurons not just
two input values but Fig. 1 can only schematically represent the initial
weights of two values as a 2D-terrian illustration. The height of a curve at
is the system fitting error $e_{i,j}$ of a particular trained network
$N(\mathbf{a}_{i},\mathbf{w}_{j})$. We use the biological term “life” to
indicate the entire process of a system’s learning.

The error-backprop learning method is a greedy method. Starting from any
initial pair $(\mathbf{a}_{i},\mathbf{w}_{j})$ it steps along the direction
that descends the quickest without knowing where the global minimum is in the
200B-dimensional space. In Fig. 1, we can see that
$(\mathbf{a}_{i},\mathbf{w}_{j})$ leads to a local minimum $a$, $b$, or $c$ if
we use the sensory mapping in Eq. (1). The point $a$ is the lowest point, but
there is no guarantee to reach it, depending on where the network starts from
in the 200B-dimensional space. The more networks have been trained, the more
likely the luckiest network finds the global minimum. In Fig. 1, the luckiest
network starts from the valley where $a$ is located, but this is much harder
for the 200B-dimensional space. Typically, the more networks a project has
trained, the more likely for the Post-Selection stage to find a network with a
smaller $e_{i^{*},j^{*}}$.

Graves et al. (Graves et al., 2016) seems to have mentioned that the number of
trained systems is at least $n=20$. Saggio et al. (Saggio et al., 2021)
reported that $n$ is at least $10,000$. Krizhevsky & Hinton (Krizhevsky et
al., 2017) did not give $n$ but seems to have mentioned 60 million parameters
which probably means each $\mathbf{w}_{i}$ and each $\mathbf{a}_{j}$ combined
to be of 60 million dimensional. Consider a small example: The number of tried
value $l$ for each hyper-parameter: $l=3$, and the number of hyper-parameters
$d=10$, the total number of hyper-parameter vectors is $k=l^{d}=3^{10}=59049$.
Letting $n=20$, $kn=l^{d}n\approx 1$M networks must be trained, a huge number
that requires a lot of computational resources to do number crunching and a
lot of manpower to manually tune the range of hyper-parameters!

###### Definition 4.1 (Distribution of fitting, validation and test errors).

The distributions of all $kn$ trained networks’ fitting errors $\\{f_{ij}\\}$,
validation errors $\\{e_{ij}\\}$, and test errors $\\{e^{\prime}_{ij}\\}$,
$i=1,2,...k$, $j=1,2,...n$ are random distributions depending on a specific
data set $D$ and its partition $D=F\cup V\cup T$. The difference between a
validation error and a test error is that the former is computed from the same
author using an author-possessed validation set $V$ but the latter is computed
by an independent agency using an author-unknown test set $T$.

We define a simple system that is easy to understand for our discussion to
follow. Consider a highly specific task of recognizing patterns inside the
annotated windows in Fig. 2. This is a simplified case of the three
tasks—recognition (yes or no, learned patterns at varied locations and
scales), detection (presence of, or not, learned patterns) and segmentation
(of recognized patterns from input). These three tasks of natural cluttered
scenes were dealt with by the first deep learning networks for 3D—Cresceptron
(Weng et al., 1997). Later data sets like ImageNet (Russakovsky et al., 2015)
contain many more image samples.

Figure 2.  Two annotated windows for an object class labeled as “steel drum”
for single object localization. Figure courtesy of (Russakovsky et al., 2015).

###  4.2. Post-Selection Illusion

To understand Post-Selection in Deep Learning (using CNN, LSTM, etc.),
including supervised learning, reinforcement learning and adversary learning,
let use consider a simple classifier that shares the same principle of Post-
Selection but does not have the distraction of details in more commonly used
networks as well as learning modes.

###### Definition 4.2 (Nearest neighbor with threshold, NNWT).

Define a network that stores the entire fitting set $F$ where each image in
$F$ may contain multiple annotated windows for matching (see Fig. 2). For each
input image $q$, a scan subwindow $x$ searches across the input image $q$ for
a range of locations and scales, normalizes the scale, and compares with each
annotated window in the fitting set $F$. Suppose an input window $x$ matches
the nearest sample (annotated) window $s$ in $F$. If the distance between $x$
and $s$ is not larger than a distance threshold $d$ (a hyper-parameter), then
the network outputs the associated label of the nearest sample $s$. Otherwise,
the system outputs a label randomly and uniformly sampled from the label set
$L$.

Let us give some formality of the Post-Selection with NNWT. Suppose that there
are three data sets, fitting set $F$, verification set $V$, and test set $T$,
a desired verification error $e_{v}\geq 0$, and a desired test error
$e_{t}\geq 0$, run the following program $P(s)$ that starts from random seed
$s$.

  1. (1)

Store all data from the fitting set $F$.

  2. (2)

For each receptive field (e.g., rectangular) $\mathbf{q}$ inside of a
cluttered image or a cluttered video $\mathbf{x}\in V$, or $\mathbf{x}\in T$,
compute the distance $d_{i}=d(\mathbf{q},\mathbf{x}_{i})$ between receptive
field $\mathbf{q}$ and each annotated receptive field $\mathbf{x}_{i}$ in $F$
for all values of $i$ (each sample in $F$ have multiple annotated receptive
field $\mathbf{x}_{i}$’s).

  3. (3)

Produce the class label $l_{j}$ of the nearest neighbor $\mathbf{x}_{j}$ that
produces the smallest distance: $j=\operatorname*{argmin}_{i}\\{d_{i}\\}$.

  4. (4)

If $d_{j}\leq d(s)$ where $d(s)$ is a threshold depending on random seed $s$,
output the label $l_{j}$ of the nearest neighbor. Otherwise, output a label
that is randomly and uniformly sampled from the label set $L$.

Post-Selection stage: If the measured verification error or the test error is
larger than required, do the above procedure $P(s)$ for a new seed $s$. Repeat
until the measured verification error and test error are both not larger than
the required $e_{v}$ and $e_{t}$, respectively.

In order words, the NNWT classifier with Post-Selections uses a lot of space
and time resources for over-fitting. It has a perfect fitting error (zero) but
it randomly guesses an output label if the distance is larger than the
threshold $d$.

To understand why Post-Selections give misleading results, let us derive the
following important theorem.

###### Theorem 4.3 (Post-Selection Illusion).

Given any validation error rate $e_{v}\geq 0$ and test error rate $e_{t}\geq
0$, using Post-Selections the NNWT classifier using Post-Selections satisfies
the $e_{v}$ and $e_{t}$, if the author is in the possession of the test set
and both the storage space and the time spent on the Post-Selections are
finite but unbounded.

###### Proof.

Because the number of seeds to be tried during the Post-Selection is
unlimited, we can prove that there is a finite time at which a lucky seed
$d(s)$ will produce the good enough verification error and test error.
Although the waiting time is long, the time is finite because $V$ and $T$ are
finite. Let us formally prove this. Suppose $l$ is the number of labels in the
output set $L$ and for the set of queries in $V$ and $T$, there are $k$
outputs that must be guessed. The probability for a single guess to be correct
is $1/l$ due to the uniform sampling in $L$. The probability for $k$ guesses
to be all correct is $(1/l)^{k}=1/l^{k}$ because guesses are independent. For
an independently and randomly initialized network to guess less than $k$ cases
correct is $1-1/l^{k}$, with $0<1-1/l^{k}<1$. The probability for as many as
$n$ independently and randomly initialized networks, all of which do not
satisfy the $e_{v}$ and $e_{t}$, is

$p(n)=(1-1/l^{k})^{n}\longrightarrow 0,\mbox{as $n\longrightarrow\infty$}$

because $0<1-1/l^{k}<1$. Therefore, within finite time span, a process of
trying incrementally more networks will get a lucky network that satisfies
both the computable $e_{v}$ and $e_{t}$. ∎

As we can see from the proof, the smaller the threshold, the more guesses must
be made and, thus, typically the longer time one needs to spend during Post-
Selections.

Let us imagine that the threshold $d(s)$ gradually increases from zero toward
infinity. The nearest neighbor classifier changes from a total-lottery scheme
(when $d(s)=0$) to a traditional nearest neighbor classifier (without label
guesses) when all query inputs are beyond the threshold.

Theorem 4.3 has established that Post-Selections can even produce a perfect
classifier that gives a zero validation error and a zero test error! Yes,
while the test sets are in the possession of authors, the authors could show
any superficially impressive validation error rates and test error rates
(including zeros!) because they used Post-Selections without a limit on
resources (to store all data sets and to search for a lucky network).

The above theorem means any comparisons without an explicit limit on storage
and time spent on Post-Selections are meaningless, like ImageNet and many
other competitions. It is of course time consuming for a program to search for
a network whose guessed labels are good enough. But such a lucky network will
eventually come within a finite time!

###### Corollary 4.4 (Misleading AI papers).

If a paper trains more than one system and the author is in the possession of
test set, the performance data from the paper are misleading if the paper does
not report the number of systems trained in Post-Selections, the amount of
computational resources (e.g., the amount of storage, the number of
computations), the amount of waiting time, along with the fitting errors, the
validation errors, and the test errors of all trained systems. The
generalization power of the reported system is still unknown.

###### Proof.

From Theorem 4.3, if Post-Selections are allowed, a NNWT can satisfy any
nonzero validation error and any nonzero testing error using Post-Selections,
since the training set, validation set and the test set are all in the
possession of the authors. The generalization power of all trained systems is
unknown, so is the reported luckiest system. ∎

This corollary is a scientific basis for the author to raise a violation of
protocols and a lack of transparency about the Post-Selection stage in almost
all machine learning papers appeared in Nature, Science, Communication of ACM
(Bengio et al., 2021) and other publication venues since around 2015. Since
all the fitting sets, validation sets and test sets are in the possession of
the authors, many papers have claimed misleading results using a flawed
protocol.

Following the terminology of (Duda et al., 2001), we use the term “Test on
Training Set” below.

###### Theorem 4.5 (Test on Training Set).

Post Selections that used test set amount to tests on training set even though
trained networks did not “see” the training set during their error backprop
training.

###### Proof.

Originally, the term “Test on Training Set” means the training stage involves
the test set. Although all trained networks did not “see” the test set, the
Post-Selection is part of the training stage that produces the reported
network and its error. Since the Post-Selection uses the test set to pick up
the luckiest network among all trained networks using their errors on the test
set, all trained networks see the test set during the post-selection stage. ∎

A typical neural network architecture has a set of hyper-parameters
represented by a vector $\mathbf{a}$, where each component corresponds a
scalar parameter, such as convolution kernel sizes and stride values at each
level of a deep hierarchy, the neuronal learning rate, and the neuronal
learning momentum value, etc. Let $k$ be a finite number of grid points along
which such hyper-parameter vectors need to be tried,
$A=\\{\mathbf{a}_{i}\;|\;i=1,2,...,k\\}$. Let’s give more detail to the above
example. Suppose there are 10 scalar parameters in each vector
$\mathbf{a}_{i}$. For each scalar parameter $x$ of the 10 hyper parameters, we
need to validate the sensitivity of the system error to $x$. With uncertainty
of $x$, we estimate its initial value as the mean $\bar{x}$, positively
perturbed estimate $\bar{x}+\sigma_{x}$ ($\sigma$ is the estimated standard
deviation of $x$), and negatively perturbed estimate $\bar{x}-\sigma_{x}$. If
each scalar hyper parameter has three values to try in this way, there are a
total of $k=3^{10}=59049$ hyper-parameter vectors to try, a very large number.
For example, in NNWT, the threshold $\bar{d}$ can be estimated by the average
of nearest distance between a sample in $V$ and the nearest neighbor in $F$
and the $\sigma_{d}$ be estimated by the standard deviation of these nearest
distances.

###  4.3. Test Data Available in Batch

Ideally, test sets should not be in the possession of authors, e.g., during a
blind test. However, this is often not true since the authors may use
publically available data sets that include test sets.

###### Definition 4.6 (Post selection in batch).

A human programmer trains multiple systems using the fitting set $F$. After
these systems have been trained, the experimenter post-selects a system by
searching, manually or assisted by computers, among trained systems based on
the batch validation set $V$ (or the batch test set $T$). This is called Post-
Selection in batch—selection of one network from multiple trained and verified
(or tested) networks.

###### Definition 4.7 (Data deletion in post-selection).

Data deletion is a misconduct during which an author deletes the performance
data of some bad-performing networks during post-selection, not reported in
the corresponding project report.

Data deletions are violations of the well-known statistical protocol of cross-
validation. In a minimally acceptable form, the distribution of the
performances of all trained networks should be reported by the worst error,
the average error and the best error among all the trained networks. Only
reporting the best error amounts to misconduct known as data deletion,
deleting the performance data of all networks that the author does not like.

Obviously, a post-selection wastes (deletes) all trained systems except the
selected one. As we can predict (Weng, 2021b), a system from the post-
selection tends to have a weaker generalization power than the reported
luckiest error indicates, as illustrated in Fig. 1.

A Post-Selection in batch can use the validation set $V$ or the test set $T$.
However, if both sets are in the possession of the human programmer, the
difference between $V$ and $T$ almost totally vanishes under Post-Selections.

###### Definition 4.8 (PSUTS and PSUVS).

A Machine PSUTS is defined as follows: If the test set $T$ is available to the
author, suppose the test error of $N(\mathbf{a}_{i},\mathbf{w}_{j})$ is
$e_{i,j}$ on the test set $T$, find the luckiest network
$N(\mathbf{a}_{i^{*}},\mathbf{w}_{j^{*}})$ so that it reaches the error of the
luckiest hyper-parameters and the luckiest initial weights from Post-Selection
Using Test Set (PSUTS):

(6) $e_{i^{*},j^{*}}=\min_{1\leq i\leq k}\min_{1\leq j\leq n}e_{i,j}$

and report only the performance $e_{i^{*},j^{*}}$ but not the performances of
other remaining $kn-1$ trained neural networks. PSUVS, V for validation, is
similarly defined.

Set $T$ is like set $V$ since it is available. Similarly, a human PSUTS is a
procedure wherein a human selects a system from multiple trained systems
according to $\\{e_{i,j}\\}$ using also human visual inspection of internal
representations of the system and their test errors.

###  4.4. Cross-Validation

The above PSUTS is an absence of cross-validation (Jain and Dubes, 1988).
Originally, the cross-validation is meant to mitigate an unfair luck in a
partition of the dataset $D$ into a fitting set $F$ and a test set $T$ (empty
validation set). For example, an unfair luck is such that every point in the
test set $T$ is well surrounded by points in the fitting set $F$. But such a
luck is hardly true in reality.

To reduce the bias of such a luck, an $n$-fold cross-validation protocol (Duda
et al., 2001) is suggested.

###  4.5. Types of Lucks in a Neural Network

In a neural network, there are at least three types of lucks:

Type-1 order lucks: The luck in a partition $P_{i}$ into a fitting set $F_{i}$
and a test set $T_{i}$ from a data set $D$ resulting in test error $e_{i}$,
$i=1,2,...,n$. Different partitions correspond to different luck outcomes.
This kind of outcome variation results in a variation of performance from
different outcomes. Conventionally, this type of lucks is filtered out by
cross-validation (e.g., $n$-fold cross-validation) as well as reporting the
deviation of $\\{e_{i}\\}$ during the cross-validation. However, such cross-
validation and deviation have hardly published for neural networks and
reported. The smaller the average $\bar{e}$ of $\\{e_{i}\\}$, the more
accurate the trained network is; the smaller the standard deviation of
$\\{e_{i}\\}$, the more trustable the average error $\bar{e}$ is.

Type-2 weights lucks: As discussed in (Weng, 2021b), weights specify the role
assignment for all the neurons in the neural network. A random seed value
determines the initialization of a pseudo-random number generator, which gives
initial weights $\mathbf{w}_{i}$ for a neural network $N(\mathbf{w}_{i})$,
resulting in a test error $e_{i}$, $i=1,2,...,n$, after training of these $n$
networks and testing on $T$. It is unknown that such a luck will be carried
over to a new test set $T^{\prime}$ that is outside the data set $D$ but was
drawn from the same distribution of $S$. Because a neural network might not
capture the internal rules of the fitting set $F$, this paper argues that a
statistical validation of the reported error should be performed by reporting
the distribution of $\\{e_{i}|i=1,2,...n\\}$, where $e_{i}$ is from a
different initial weight vector $\mathbf{w}_{i}$. For example, Krizhevsky et
al. (Krizhevsky et al., 2017) reported 60 million parameters, mostly in
$\mathbf{w}_{i}$ but only the luckiest $e_{i}$ was reported. The smaller the
average $\bar{e}$ of $\\{e_{i}\\}$, the more accurate the trained network is;
the smaller the standard deviation $\sigma$ of $\\{e_{i}\\}$, the less
sensitive the trained neural network is to the initial weights and thus the
accuracy is more trustable for real applications. For i.i.d. (identically
independently distributed) errors, we can expect that doubling the number $n$
will reduce the expected variance of $\bar{e}$ by a factor $1/\sqrt{2}$.,
since the expected variance of $n$ random numbers is about $\sigma^{2}/n$.

Type-3 hyper-parameter lucks: Each hyper-parameter vector $\mathbf{a}_{j}$ of
the neural network gives an error $e_{j}$, $j=1,2,...,k$. Because such a luck
of $\mathbf{a}_{j}$ might not capture the internal rules of the fitting set
$F_{j}$, this paper argues that a statistical validation of the reported error
estimate should be performed and the distribution of $\\{e_{j}\\}$ be
reported. In our above example, the number of distinct hyper-parameter vectors
to be tried is $k=3^{10}=59049$. The smaller the average $\bar{e}$ of
$\\{e_{j}\\}$, the more accurate the trained network is; the smaller the
sample variance of $\\{e_{j}\\}$, the more trustable $\bar{e}$ is, namely, the
average error $\bar{e}$ is less sensitive to the initial hyper-parameters of
the network. For example, the threshold $d$ of the nearest neighbor classifier
in Definition 4.2 might result in a large deviation. A good way is to reduce
the manual selection nature of such hyper-parameters. For example, all hyper-
parameters are adaptively adjusted from the initial hyper-parameters that are
further automatically computed from system resources, e.g., the resolution of
a camera, the total number of available neurons, and the firing age of each
neuron (Weng and Luciw, 2009).

For notation clarity in the discussion that follows, index $j$ is used in Type
3 to distinguish index $i$ in type 2, but the above three types of lucks are
all different.

Let us discuss the case of a developmental network, such as Cresceptron (Weng
et al., 1997) and DN (Weng, 2015). Type-1 cross-validation is not needed
because of reporting of a lifetime error. In other words, errors of all new
tests in each life are taken into account throughout the lifetime. Type-2
validation is not needed because all different random weights $\mathbf{w}_{i}$
leads to the function-equivalent neural network under certain conditions. For
example, in top-$k$ competition, with $k=1$ different $\mathbf{w}_{i}$ give
the exactly the same neural network and with $k>1$ different $\mathbf{w}_{i}$
give almost the same neural network. The distribution of lifetime errors
$\\{e_{i}\\}$ is expected to have a negligible deviation across different
initial weight vectors $\mathbf{w}_{i}$, given the same Four Learning
Conditions. Type-3 validation might be useful but is expected to be negligible
since the most obvious parameters such as learning rate and momentum of
learning rate is automatically and optimally determined by each neuron, not
handcrafted, as in LCA (Weng and Luciw, 2009). The synaptic maintenance
automatically adjusts all receptive fields (Wang et al., 2011; Guo et al.,
2014) so that the neural network performance is not sensitive to the initial
hyper-parameters.

In contract, a batch-trained neural network typically uses a Post-Selection to
pick the luckiest network without cross-validation for either of the above
three types of lucks, e.g., in ImageNet Contest (Russakovsky et al., 2015).
Namely, errors occurred during batch training of the network before the
network is finalized and how long the training takes are not reported. Many
researchers have claimed error-backprop “works” without providing much-needed
three types of validations. This seems not true since (Zheng and Weng, 2016)
shows a huge difference between the luckiest CNN with error-backprop and the
optimal DN.

We also need to be aware of another protocol flaw: Random seeds and hyper
parameters are all coupled. Under such a coupling, Type 2 validation seems
unnecessary with $n=1$ but the search of the luckiest weights is embedded into
the search for the luckiest hyper-parameter vector. where each hyper parameter
vector uses a different seed.

Since a PSUTS procedure picks the best system based on the errors on the test
set (like a validation set), the resulting system does not do well on new test
sets because doing well on a validation set does not guarantee doing well on
an open test set. See Fig. 1. Typically, due to a very large number of
samples, availability of test sets and unavailability of open test sets in a
properly managed contest, Post-Selections cause the reported error to be
smaller than an open test error rate. (However, in Table 2 of (Krizhevsky et
al., 2017), the test error rate is smaller than the validation error for
7CNNs, causing a reasonable suspicion that PSUTS could be used in addition to
PSUVS.)

###  4.6. The Luckiest Network from a Validation Set: Data Deletion

Many people may ask: Are there any technical flaws in at least PSUVS, since it
does not use the test sets? Any post-selection is technically flawed and
results in misleading results, including both PSUVS and PSUTS, because of the
data deletion misconduct.

In general, Type-1 cross-validation is to filter out lucks in data partition
that a typical user does not have during a deployment of the method. Namely,
it is a severe technical and protocol flaw in reporting only the luckiest
network, regardless the post-selection uses validation sets or test sets. At
least the average error over Post-Selections must be reported.

This conclusion has a great impact on evolutional methods that often report
only the luckiest network, instead of those of all networks in a population.
Namely, the performances of all individual networks in an evolutionary
generation should be reported. Furthermore, a reasonably disjoint test set
must be used to evaluate the generalization of the luckiest network.

If the test set $T$ is in the possession of the author, which seems to be true
for almost all so-called “deep learning” publications other than open
competitions, we define machine PSUTS as a Post-Selection process where test
set is used, in addition to the validation set.

Some researchers have claimed that the test data was “unseen” by trained
systems when they were tested, but the network selector has seen their
performances when they see the test set. In other words, the Post-Selection
stage belongs to the training phase—training the Post-Selector.

Weng (Weng, 2021b) discussed also human PSUTS where it is a human that does
Post-Selection. The subsection below discusses human PSUTS on the fly, not in
batch.

Weng (Weng, 2021b) also discussed why error-backprop needs machine PSUTS and
human PSUTS.

###  4.7. Open Test Data Arriving on the Fly

###### Definition 4.9 (PSUTS On The Fly).

During an open competition, the participating $m>1$ luckiest networks meet new
open test data that arrive on the fly. One or more humans conduct an
additional round of Post-Selections on the fly during the competition using
human PSUTS from the $m$ machine outputs for some actions. Here, all human
selected actions across the entire game do not even have to be consistently
from a single network.

For example, $\mathbf{x}(t)$ is the current board configuration in a Go game,
treated as a game state at player time $t$. Human experts (behind the scene or
on the scene), based on the output actions from the $m$ luckiest networks,
manually select a network’s output as the next action $\mathbf{z}(t+1)$. Of
course, this is not a fair competition, because the so-called “human player
side” has only a single human who is not allowed to use any computer but the
so-called “machine player side” has one or more humans who have assistance
from computers.

Weng (Weng, 2021a) discussed how the DN algorithm trains only a single network
that is always optimal in the sense of maximum likelihood, conditioned on the
Four Learning Conditions. In this sense, the DN is free from local minima.

##  5\. Conclusions

We used intuitive terms but formal ways to discuss Post-Selections. So-called
“Deep Learning” involves two misconducts, “data deletion” and “test on
training data”. Because it is without a test stage, “Deep Learning” is not
generalizable. Performance data from so-called “deep learning” are misleading
without explicit exclusion of such flaws in their experimental protocol. Such
flawed protocols are tempting to those published papers where the test sets
are in the possession of the authors and also to open-competitions where human
experts are not explicitly disallowed to interact with the “machine player” on
the fly. Evidence of such misconduct is referred to Weng et al. v. NSF et al.
MWDC 1:22-cv-998.

Public and media have gained an impression that deep learning has approached
or even “sometimes exceeded” human level performance on certain tasks. For
example, the image classification errors from a static image set were compared
with those of humans (Russakovsky et al., 2015, A2, p242)) and the work is
laudable. However, this paper raises Post-Selections, which seem to question
such claims since a real human does not have the luxury of Post-Selections.
The author hopes that the exposure of Post-Selections is beneficial to AI
credibility and the future healthy development of AI, especially with the
concepts of developmental errors and the framework of ML-optimal lifetime
learning for invariant concepts under the Four Learning Conditions. Some
researchers have raised that it seems that those who wan a competition were
those who have more computational resources and manpower at their disposal.
The new developmental error metrics under the Four Learning Conditions
hopefully encourages future AI competitions to compare methods under the same
Four Learning Conditions. Considering DN as a much-simplified model for a
biological machine, it seems not baseless to guess that each biological brain
is probably ML-optimal (of course in a much richer sense) across lifetime,
e.g., due to the pressure to compete at every age. The Four Learning
Conditions explicitly include other factors that greatly affect machine
learning performances such as learning framework (e.g., task-nonspecificity,
incremental learning, the robot bodies), learning experiences and
computational resources. The analysis that any “big data” sets are nonsalable
does not mean that we should not create, use and share data sets. Instead, we
need to pay attention to the fundamental limitations of any static data sets,
regardless how large their apparent sizes are.

## References

  * (1)
  * Bellemare et al. (2020) M. G. Bellemare, S. Candido, J. Gong P. S. Castro, M. C. Machado, S. Moitra, S. S. Ponda, and Z. Wang. 2020.  Autonomous navigation of stratospheric balloons using reinforcement learning.  _Nature_ 588, 7836 (2020), 77–82.
  * Bengio et al. (2021) Y. Bengio, Y. LeCun, and G. Hinton. 2021.  Deep Learning for AI.  _Communications of ACM_ 64, 7 (2021), 58–65.
  * Duda et al. (2001) R. O. Duda, P. E. Hart, and D. G. Stork. 2001.  _Pattern Classification_ (2nd ed.).  Wiley, New York.
  * et al. (2015) V. Mnih et al. 2015\.  Human-level control through deep reinforcement learning.  _Nature_ 518 (2015), 529–533.
  * Funahashi (1989) K. I. Funahashi. 1989\.  On the Approximate Realization of Continuous Mappings by Neural Networks.  _Neural Network_ 2, 2 (March 1989), 183–192.
  * Graves et al. (2016) A. Graves, G. Wayne, M. Reynolds, T. Harley, I. Danihelka, A. Grabska-Barwinska, S. G. Colmenarejo, E. Grefenstette, T. Ramalho, J. Agapiou, A. P. Badia, K. M. Hermann, Y. Zwols, G. Ostrovski, A. C., H. King, C. Summerfield, P. Blunsom, K. Kavukcuoglu, and D. Hassabis. 2016.  Hybrid computing using a neural network with dynamic external memory.  _Nature_ 538 (2016), 471–476.
  * Guo et al. (2014) Q. Guo, X. Wu, and J. Weng. 2014.  WWN-9: Cross-Domain Synaptic Maintenance and Its Application to Object Groups Recognition. In _Proc. Int’l Joint Conference on Neural Networks_. IEEE Press, Beijing, China, 1–8.
  * Iverson (2010) J. M. Iverson. 2010\.  Developing language in a developing body: the relationship between motor development and language development.  _Journal of child language_ 37, 2 (2010), 229–261.
  * Jain and Dubes (1988) A. K. Jain and R. C. Dubes. 1988.  _Algorithms for Clustering Data_.  Prentice-Hall, New Jersey.
  * Krizhevsky et al. (2017) A. Krizhevsky, I. Sutskever, and G. E. Hinton. 2017\.  Imagenet classification with deep convolutional neural networks.  _Commun. ACM_ 60, 6 (2017), 84–90.
  * LeCun et al. (2015) Y. LeCun, L. Bengio, and G. Hinton. 2015.  Deep Learning.  _Nature_ 521 (2015), 436–444.
  * McKinney et al. (2020) S. M. McKinney, M. Sieniek, V. Godbole, J. Godwin, N. Antropova, H. Ashrafian, T. Back, M. Chesus, G. S. Corrado, A. Darzi, M. Etemadi, F. Garcia-Vicente, F. J. Gilbert, M. Halling-Brown, D. Hassabis, S. Jansen, A. Karthikesalingam, C. J. Kelly, D. King, J. R. Ledsam, D. Melnick, H. Mostofi, L. Peng, J. J. Reicher, B. Romera-Paredes, R. Sidebottom, M. Suleyman, D. Tse, K. C. Young, J. De Fauw, and S. Shetty. 2020.  Int’l evaluation of an AI system for breast cancer screening.  _Nature_ 577 (2020), 89–94.
  * Moore et al. (2003) T. Moore, K. M. Armstrong, and M. Fallah. 2003\.  Visuomotor origins of covert spatial attention.  _Neuron_ 40 (2003), 671–683.
  * Moravcik et al. (2017) M. Moravcik, M. Schmid, N. Burch, V. Lisy, D. Morrill, N. Bard, T. Davis, K. Waugh, M. Johanson, and M. Bowling. 2017.  DeepStack: Expert-level artificial intelligence in heads-up no-limit poker.  _Science_ 356 (2017), 508–513.
  * Poggio and Girosi (1990) T. Poggio and F. Girosi. 1990.  Networks for approximation and learning.  _Proceedings of The IEEE_ 78, 9 (1990), 1481–1497.
  * Rizzotti et al. (1987) G. Rizzotti, L. Riggio, I. Dascola, and C. Umilta. 1987\.  Reorienting attention across the horizontal and vertical meridians: evidence in favor of a premotor theory of attention.  _Neuropsychologia_ 25 (1987), 31–40.
  * Roy (2008) A. Roy. 2008.  Connectionism, Contollers, and a Brain Theory.  _IEEE Trans. on System, Man, and Cybernetics—Part A; Systems and Humans_ 38, 6 (Nov. 2008), 1434–1441.
  * Russakovsky et al. (2015) O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei. 2015.  ImageNet Large Scale Visual Recognition Challenge.  _Int’l Journal of Computer Vision_ 115 (2015), 211–252.
  * Saggio et al. (2021) V. Saggio, B. E. Asenbeck, A. Hamann, T. Stromberg, P. Schiansky, V. Dunjko, N. Friis, N. C. Harris, M. Hochberg, D. Englund, S. Wolk, H. J. Briegel, and P. Walther. 2021.  Experimental quantum speed-up in reinforcement learning agents.  _Nature_ 591, 7849 (March 11 2021), 229–233.
  * Schrittwieser et al. (2020) J. Schrittwieser, I. Antonoglou, T. Hubert, K. Simonyan, L. Sifre, S. Schmitt, A. Guez, E. Lockhart, D. Hassabis, T. Graepel, T. Lillicrap, and D. Silver. 2020.  Mastering Atari, Go, chess and shogi by planning with a learned model.  _Science_ 588, 7839 (2020), 604–609.
  * Senior et al. (2020) A. W. Senior, R. Evans, J. Jumper, J. Kirkpatrick, L. Sifre, T. Green, C. Qin, A. Zidek, A. W. R. Nelson, A. Bridgland, H. Penedones, S. Petersen, K. Simonyan, S. Crossan, P. Kohli, D. T. Jones, D. Silver, K. Kavukcuoglu, and D. Hassabis. 2020\.  Improved protein structure prediction using potentials from deep learning.  _Nature_ 577 (2020), 706–710.
  * Silver et al. (2016) D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe, J. Nham, N. Kalchbrenner, I. Sutskever, T. Lillicrap, M. Leach, K. Kavukcuoglu, T. Graepel, and D. Hassabis. 2016\.  Mastering the game of Go with deep neural networks and tree search.  _Nature_ 529 (January 27 2016), 484–489.
  * Silver et al. (2018) D. Silver, T. Hubert, J. Schrittwieser, I. Antonoglou, M. Lai, A. Guez, M. Lanctot, L. Sifre, D. Kumaran, T. Graepel, T. Lillicrap, K. Simonyan, and D. Hassabis. 2018\.  A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play.  _Science_ 362, 6419 (2018), 1140–1144.
  * Silver et al. (2017) D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, A. Bolton, Y. Chen, T. Lillicrap, F. Hui, L. Sifre, G. van den Driessche, T. Graepel, and D. Hassabis. 2017\.  Mastering the game of Go without human knowledge.  _Nature_ 550 (2017), 354–359.
  * Super (1976) C. M. Super. 1976\.  Environmental effects on motor development: A case of Africa infant precocity.  _Developmental Medicine and Child Neurology_ 18 (1976), 561–567.
  * Thoroughman and Taylor (2005) K. A. Thoroughman and J. A. Taylor. 2005.  Rapid Reshaping of Human Motor Generalization.  _Jounal of Neuroscience_ 25, 39 (2005), 8948–8953.
  * Wang et al. (2011) Y. Wang, X. Wu, and J. Weng. 2011.  Synapse Maintenance in the Where-What Network. In _Proc. Int’l Joint Conference on Neural Networks_. Springer, San Jose, CA, 2823–2829.
  * Weng (2011) J. Weng. 2011.  Why Have We Passed “neural networks do not abstract well”?  _Natural Intelligence: the INNS Magazine_ 1, 1 (2011), 13–22.
  * Weng (2015) J. Weng. 2015.  Brain as an Emergent Finite Automaton: A Theory and Three Theorems.  _Int’l Journal of Intelligence Science_ 5, 2 (2015), 112–131.
  * Weng (2020a) J. Weng. 2020a.  Autonomous Programming for General Purposes: Theory.  _Int’l Journal of Huamnoid Robotics_ 17, 4 (August 2020), 1–36.
  * Weng (2020b) J. Weng. 2020b.  Conscious Intelligence Requires Developmental Autonomous Programming For General Purposes. In _Proc. IEEE Int. Conf. on Dev. Learning and Epigenetic Robotics_. IEEE Press, Valparaiso, Chile, 1–7.
  * Weng (2021a) J. Weng. 2021a.  A Developmental Method that Computes Optimal Networks without Post-Selections. In _Proc. IEEE Int’l Conference on Development and Learning_. IEEE Press, Beijing, China, 1–6.
  * Weng (2021b) J. Weng. 2021b.  On Post Selections Using Test Sets (PSUTS) in AI. In _Proc. Int’l Joint Conference on Neural Networks_. IEEE Press, Shenzhen, China, 1–8.
  * Weng et al. (1997) J. Weng, N. Ahuja, and T. S. Huang. 1997.  Learning recognition and segmentation using the Cresceptron.  _Int’l Journal of Computer Vision_ 25, 2 (Nov. 1997), 109–143.
  * Weng and Luciw (2009) J. Weng and M. Luciw. 2009\.  Dually Optimal Neuronal Layers: Lobe Component Analysis.  _IEEE Trans. Autonomous Mental Development_ 1, 1 (2009), 68–85.
  * Weng and Luciw (2012) J. Weng and M. Luciw. 2012\.  Brain-Like Emergent Spatial Processing.  _IEEE Trans. Autonomous Mental Development_ 4, 2 (2012), 161–185.
  * Weng et al. (2013) J. Weng, M. Luciw, and Q. Zhang. 2013.  Brain-Like Temporal Processing: Emergent Open States.  _IEEE Trans. Autonomous Mental Development_ 5, 2 (2013), 89 – 116.
  * Weng et al. (2020) J. Weng, Zejia Zheng, Xiang Wu, and Juan Castro-Garcia. 2020\.  Auto-Programming for General Purposes: Theory and Experiments. In _Proc. Int’l Joint Conference on Neural Networks_. IEEE Press, Glasgow, UK, 1–8.
  * Weng et al. (2018) J. Weng, Z. Zheng, X. Wu, J. Castro-Garcia, S. Zhu, Q. Guo, and X. Wu. 2018.  Emergent Turing Machines and Operating Systems for Brain-Like Auto-Programming for General Purposes. In _Proc. AAAI 2018 Fall Symposium: Gathering for AI and Natural Systems_. AAAI Press, Arlington, Virginia, 1–7.
  * Zheng and Weng (2016) Z. Zheng and J. Weng. 2016\.  Mobile Device Based Outdoor Navigation With On-line Learning Neural Network: a Comparison with Convolutional Neural Network. In _Proc. 7th Workshop on Computer Vision in Vehicle Technology (CVVT 2016) at CVPR 2016_. IEEE Press, Las Vegas, 11–18.