# One-Trial Correction of Legacy AI Systems and Stochastic Separation Theorems

Alexander N. Gorban<EMAIL_ADDRESS>Richard Burton<EMAIL_ADDRESS>Ilya Romanenko<EMAIL_ADDRESS>Ivan Yu. Tyukin111The work was
supported by Innovate UK Technology Strategy Board (Knowledge Transfer
Partnership grants KTP009890 and KTP010522). The corresponding author is
partially supported by the RFBR grant 15-38-20178<EMAIL_ADDRESS>University of Leicester, Department of Mathematics, University Road,
Leicester, LE1 7RH, UK  Department of Automation and Control Processes, St.
Petersburg State University of Electrical Engineering, Prof. Popova str. 5,
Saint-Petersburg, 197376, Russian Federation  Imaging and Vision Group, ARM
Ltd, 1 Summerpool Rd, Loughborough, LE11 5RD, UK

###### Abstract

We consider the problem of efficient “on the fly” tuning of existing, or
legacy, Artificial Intelligence (AI) systems. The legacy AI systems are
allowed to be of arbitrary class, albeit the data they are using for computing
interim or final decision responses should posses an underlying structure of a
high-dimensional topological real vector space. The tuning method that we
propose enables dealing with errors without the need to re-train the system.
Instead of re-training a simple cascade of perceptron nodes is added to the
legacy system. The added cascade modulates the AI legacy system’s decisions.
If applied repeatedly, the process results in a network of modulating rules
“dressing up” and improving performance of existing AI systems. Mathematical
rationale behind the method is based on the fundamental property of measure
concentration in high dimensional spaces. The method is illustrated with an
example of fine-tuning a deep convolutional network that has been pre-trained
to detect pedestrians in images.

###### keywords:

Measure concentration, separation theorems, big data, machine learning.

##  1 Introduction

Legacy information systems, i.e. information systems that have already been
created and form crucial parts of existing solutions or service [3], [4], are
common in engineering practice and scientific computing [12]. They are
becoming particularly wide-spread, as legacy Artificial Intelligence (AI)
systems, in the areas of computer vision, image processing, data mining, and
machine learning (see e.g. Caffe [26], MXNet [7] and Deeplearning4j [43]
packages).

Despite their success and utility, legacy AI systems occasionally make
mistakes. Their generalization errors may be caused by a number of issues, for
example, by incomplete, insufficient or obsolete information used in their
development and design, or by oversensitivity to some signals, etc.
Adversarial images [42], [34] are well-known examples of such issues for
classifiers based on Deep Learning Convolutional Neural Networks (CNNs) [28].
It has been shown in [42] that imperceptible changes in the images used in the
training set may cause labelling errors, and at the same time completely
unrecognizable to human perception objects can be classified as the ones that
have already been learnt [34]. Retraining such legacy systems requires
resources, e.g. computational power, time and/or access to training sets, that
are not always available. Thus solutions that reliably correct mistakes of
legacy AI systems without retraining are needed.

Significant progress has been made to date to understand and mitigate atypical
and spurious mistakes. For example, in [20], [22], [23] using ensembles of
classifiers was shown to improve performance of the overall system. With
regards to adversarial images, augmenting data [29], [33], [35] and enforcing
continuity of feature representations [47] help to increase reliability of
classification. These approaches nevertheless do not warrant error-free
behavior; AI and machine learning systems drawing conclusions on the basis of
empirical data are expected to make mistakes, as human experts occasionally do
too [11]. Hence having a method for fast and reliable rectification of these
errors in legacy systems is crucial.

In this work we present a computationally efficient solution to the problem of
correcting AI systems’ mistakes. In contrast to conventional approaches [29],
[33], [35], [47] focusing on altering data and improving design procedures of
legacy AI systems, we propose that the legacy AI system itself be augmented by
miniature low-cost and low-risk additions. These additions are, in their
essence, small neural network cascades that in principle can be easily
incorporated into existing architectures already employing neural networks as
their inherent components. Importantly, we show that for a large class of
legacy AI systems in which decision criteria are calculated on the basis of
high-dimensional data representation, such small neuronal cascades can be
constructed via simple non-iterative procedures. We prove this by showing that
in an essentially high-dimensional finite random set with probability close to
one all points are extreme, i.e. every point is linearly separable from the
rest of the set. Such a separability holds even for large random sets up to
some upper limit, which grows exponentially with dimension. This is proven for
finite samples i.i.d. drawn from an equidistribution in a ball or ellipsoid
and demonstrated also for equidistributions in a cube and for normal
distribution. We can hypothesize now that this is a very general property of
all sufficiently regular essentially high dimensional distributions.

Hence, if a legacy AI system employs high-dimensional internal data
representation then a single element, i.e. a mistake, in this representation
can be separated from the rest by a simple perceptron, for example, by the
Fisher discriminant. Several such mistakes can be collated into a common
corrector implementable as a small neuronal cascade. The results are based on
ideas of measure concentration [18], [19], [15] and can be related to the
works of Gibbs [13] and Levi [30] and, on the other hand, to the Krein-Milman
theorem.

The paper is organized as follows. Section 2 presents formal statement of the
problem, Section 3 contains mathematical background for developing One-Trial
correctors of Legacy AI systems, in Section 4 we state and discuss practically
relevant generalizations, Section 5 provides experimental results showing
viability of our method for correcting state-of-the-art Deep Learning CNN
classifiers, and Section 6 concludes the paper. Main notational agreements are
summarized in the Notation section below.

## Notation

Throughout the paper the following notational agreements are used.

  * 1.

$\mathbb{R}$ denotes the field of real numbers;

  * 2.

$\mathbb{N}$ is the set of natural numbers;

  * 3.

$\mathbb{R}^{n}$ stands for the $n$-dimensional real space; unless stated
otherwise symbol $n$ is reserved to denote dimension of the underlying linear
space;

  * 4.

let $x\in\mathbb{R}^{n}$, then $\|x\|$ is the Euclidean norm of $x$:
$\|x\|=\sqrt{x_{1}^{2}+\cdots+x_{n}^{2}}$;

  * 5.

$B_{n}(R)$ denotes a $n$-ball of radius $R$ centered at $0$:
$B_{n}(R)=\\{x\in\mathbb{R}^{n}|\ \|x\|\leq R\\}$;

  * 6.

$\mathcal{V}(\Xi)$ is the Lebesgue volume of $\Xi\subset\mathbb{R}^{n}$;

  * 7.

$\mathcal{M}$ is an i.i.d. sample equidistributed in $B_{n}(1)$;

  * 8.

$M$ is the number of points in $\mathcal{M}$, or simply the cardinality of the
set $\mathcal{M}$.

##  2 Problem Formulation

Consider a legacy AI system that processes some input signals, produces
internal representations of the input and returns some outputs. For the
purposes of this work the exact nature and definition of the process and
signals themselves are not important. Input signals may correspond to any
physical measurements, outputs could be alarms or decisions, internal
representations could include but not limited to extracted features, outputs
of computational sub-routines etc. In the case of a CNN classifier inputs are
images, internal signals are outputs of convolutional and dense layers, and
outputs are labels of objects.

We assume, however, that some relevant information about the input, internal
signals, and outputs can be combined into a common object, $x$, representing,
but not necessarily defining, the state of the AI system. The objects $x$ are
assumed to be elements of $\mathbb{R}^{n}$. Over relevant period of activity
the AI system generates a set $\mathcal{M}$ of representations $x$. Since we
do not wish to impose any prior knowledge of how $x$ are constructed, it is
natural to assume that $\mathcal{M}$ is a random sample drawn from some
meaningful distribution.

We suppose that some elements of the set $\mathcal{M}$ are labelled as those
corresponding to “errors” or mistakes which the original legacy AI system
made. The task is to single out these errors and augment the AI systems’
response by an additional device, a corrector. A diagram illustrating this
setting is shown in Fig. 1.

Figure 1: Corrector of Legacy AI systems

The corrector system must in turn satisfy a list of properties ensuring that
its deployment is practically feasible

  1. 1.

the elements comprising the corrector must be re-usable in the original AI
system;

  2. 2.

the elements, as elementary learning machines, should be able to generalize;

  3. 3.

the elements must be efficient computationally;

  4. 4.

the corrector must allow for fast non-iterative learning;

  5. 5.

the corrector should not affect functionality of the AI system, for reasonably
large $\mathcal{M}$ with $|\mathcal{M}|\gg n$.

For a broad range of AI systems, a candidate element satisfying requirements
$1)-3)$ is the parameterized affine functional

$l(x)=\langle x,w\rangle+b,\ w\in\mathbb{R}^{n},\ b\in\mathbb{R}$ (1)

followed, if needed, by a suitable nonlinearity. It is a major building block
of neural networks (convolutional, deep and shallow) as well as in decision
trees and support vector machines. It’s computational efficiency and
generalization capabilities [44] are well-known too. Whether remaining
requirements could be guaranteed, however, is not clear. The problem therefore
is to find an answer to this question.

In the next sections we show that, remarkably, in high dimensions the affine
functionals do have these properties, with high probability.

##  3 Mathematical Background

###  3.1 Separation of single points in finite random sets in high dimensions

Our basic example throughout this section is an equidistribution in the unit
ball222Partially, the ideas concerning equidistributions in $B_{n}(1)$ have
been presented in a conference talk [17] $B_{n}(1)$ in $\mathbb{R}^{n}$.
Genralizations to equidistribution in an ellipsoid will be formulated in
Section 4.1, and other distributions will be discussed in Sections 4.2 and 5.

###### Definition 1

Let $X$ and $Y$ be subsets of $\mathbb{R}^{n}$. We say that a linear
functional $l$ on $\mathbb{R}^{n}$ separates $X$ and $Y$ if there exists a
$t\in\mathbb{R}$ such that

$l(x)>t>l(y)\ \forall\ x\in X,\ y\in Y.$

Let $\mathcal{M}$ be an i.i.d. sample drawn from the equidistribution on the
unit ball $B_{n}(1)$. We begin with evaluating the probability that a single
element $x$ randomly and independently selected from the same equidistribution
can be separated from $\mathcal{M}$ by a linear functional. This probability,
denoted as $P_{1}(\mathcal{M},n)$, is estimated in the theorem below.

###### Theorem 1

Consider an equidistribution in a unit ball $B_{n}(1)$ in $\mathbb{R}^{n}$,
and let $\mathcal{M}$ be an i.i.d. sample from this distribution. Then

$\begin{split}P_{1}(\mathcal{M},n)&\geq\max_{\varepsilon\in(0,1)}(1-(1-\varepsilon)^{n})\left(1-\frac{{\rho}(\varepsilon)^{n}}{2}\right)^{M},\\\
{\rho}(\varepsilon)&=(1-(1-\varepsilon)^{2})^{\frac{1}{2}}\end{split}$ (2)

Proof of Theorem 1. The proof of the theorem is contained mostly in the
following lemma

###### Lemma 1

Let $y$ be random point from an equidistribution on a unit ball $B_{n}(1)$.
Let $x\in B_{n}(1)$ be a point inside the ball with $1>\|x\|>1-\varepsilon>0$.
Then

$P\left(\left\langle\frac{x}{\|x\|},y\right\rangle<1-\varepsilon\right)\geq
1-\frac{{\rho}(\varepsilon)^{n}}{2}.$ (3)

Proof of Lemma 1. Recall that [30]:
$\mathcal{V}(B_{n}(r))=r^{n}\mathcal{V}(B_{n}(1))$ for all $n\in\mathbb{N}$
$r>0$. The point $x$ is inside the spherical cap $C_{n}(\varepsilon)$:

$C_{n}(\varepsilon)=B_{n}(1)\cap\left\\{\xi\in\mathbb{R}^{n}\left|\
\left\langle\frac{x}{\|x\|},\xi\right\rangle>1-\varepsilon\right\\}\right..$
(4)

The volume of this cap can be estimated from above [2] (see Fig. 2) as

$\mathcal{V}(C_{n}(\varepsilon))\leq\frac{1}{2}\mathcal{V}(B_{n}(1))\rho(\varepsilon)^{n}.$
(5)

The probability that the point $y\in\mathcal{M}$ is outside of
$C_{n}(\varepsilon)$ is equal to
$1-\mathcal{V}(C_{n}(\varepsilon))/\mathcal{V}(B_{n}(1))$. Estimate (3) now
immediately follows from (5). $\square$

Figure 2: A test point $x$ and a spherical cap on distance $\varepsilon$ from
the surface of unit sphere. Escribed sphere of radius $\rho$ is showed by
dashed line.

Let us now return to the proof of the theorem. If $x$ is selected
independently from the equidistribution on $B_{n}(1)$ then the probabilities
that $x=0$ or that it is on the boundary of the ball are $0$. Let $x\neq 0$ be
in the interior of $B_{n}(1)$. According to Lemma 1, the probability that a
linear functional $l$ separates $x$ from a point $y\in\mathcal{M}$ is larger
than $1-1/2\rho(\varepsilon)^{n}$. Given that points of the set $\mathcal{M}$
are i.i.d. in accordance to the equidistribution on $B_{n}(1)$, the
probability that $l$ separates $x$ from $\mathcal{M}$ is no smaller than
$(1-1/2\rho(\varepsilon)^{n})^{M}$.

On the other hand

$P(1>\|x\|>1-\varepsilon\,|\,x\in B_{n}(1))=(1-(1-\varepsilon)^{n}).$

Given that $x$ and $y\in\mathcal{M}$ are independently drawn from the same
equidistribution and that the probabilities of randomly selecting the point
$x$ exactly on the boundary of $B_{n}(1)$ or in its centre are zero, we can
conclude that

$P_{1}(\mathcal{M},n)\geq(1-(1-\varepsilon)^{n})(1-1/2\rho(\varepsilon)^{n})^{M}.$
(6)

Finally, noticing that (6) holds for all $\varepsilon\in(0,1)$ including the
value of $\varepsilon$ maximizing the rhs of (6), we can conclude that (2)
holds true too. $\square$

###### Remark 1

For $\rho(\varepsilon)^{n}$ small (i.e. $\rho(\varepsilon)^{n}\ll 1$) the term
$\left(1-\frac{\rho^{n}}{2}\right)^{M}$ can be approximated as
$(1-\rho^{n}/2)^{M}\approx e^{-M\frac{\rho^{n}}{2}}$. Thus estimate (2)
becomes

$P_{1}(\mathcal{M},n)\gtrsim\max_{\varepsilon\in(0,1)}(1-(1-\varepsilon)^{n})e^{-M\frac{\rho^{n}}{2}},\
\rho(\varepsilon)^{n}\ll 1.$ (7)

To see how large the probability $P_{1}(\mathcal{M},n)$ could become for
already rather modest dimensions, e.g. for $n=50$, we estimate the right-hand
side of (7) from below by letting $\varepsilon=1/5$ and $\rho=3/5$:

$P_{1}(\mathcal{M},50)\gtrsim 0.99998\exp(-4\times 10^{-12}M).$

For $M\leq 10^{9}$ this estimate gives $P_{1}(M,50)\gtrsim 0.996$. Thus in
dimension 50 (and higher) a random point is linearly separable from a random
set of $10^{9}$ points with probability $0.996$.

###### Remark 2

If $x$ is an element from the sample $\mathcal{M}$ then the probability $p$
that $x$ is linearly separable from all other points in the sample is bounded
from below by $P_{1}(\mathcal{M},n)$.

###### Remark 3

Let $x\in\mathcal{M}$ be a given query point. This query point determines the
value of $\varepsilon=1-\|x\|$ as the least $\varepsilon$-thickening of the
unit sphere containing $x$. With probability $1$ the values of $\varepsilon$
belong to the open interval $(0,1)$. Let $p\in(0,1)$ be the desired
probability that $x$ is separated from the rest of the sample $\mathcal{M}$.
It is clear that the estimate $P_{1}(\mathcal{M},n)\geq p$ holds for $M$ from
some interval $[1,\overline{M}]$. Interestingly, for $n$ large enough, the
maximal number $\overline{M}$ is exponentially large in dimension $n$. Indeed,
let us fix the values of $\varepsilon\in(0,1)$ and $p\in(0,1)$. Then we find
the estimate of the maximal possible sample size for which
$P_{1}(\mathcal{M},n)\geq p$ remains valid:

$\max\\{\overline{M}\\}\geq\frac{\ln(p)}{\ln\left(1-\frac{\rho(\varepsilon)^{n}}{2}\right)}-\frac{\ln(1-(1-\varepsilon)^{n})}{\ln\left(1-\frac{\rho(\varepsilon)^{n}}{2}\right)}$

Using

$\frac{x}{x-1}\leq\ln(1-x)\leq-x$

we conclude that

$\max\\{\overline{M}\\}\geq\left(\frac{1}{\rho(\varepsilon)}\right)^{n}C(n,\varepsilon)\\\
$

where

$C(n,\varepsilon)=2\left(|\ln(p)|\left(1-\frac{\rho(\varepsilon)^{n}}{2}\right)-|\ln(1-(1-\varepsilon)^{n})|\right).$

Observe that for any fixed $\varepsilon\in(0,1)$ there is an $N(\varepsilon)$
large enough such that $C(n,\varepsilon)\geq|\ln(p)|$ for all $n\geq
N(\varepsilon)$. Hence, for $n$ sufficiently large the following estimate
holds:

$\max\\{\overline{M}\\}\geq
e^{n\ln\left(\rho(\varepsilon)^{-1}\right)}|\ln(p)|.$ (8)

Equation (8) can be viewed as a separation capacity estimate of linear
functionals. This estimate links the level of desired performance specified by
$p$, maximal size of the sample, $M$, and parameters of the data, $n$ and
$\varepsilon$.

###  3.2 Extreme points of a random finite set

So far we have discussed the question of separability of a single random point
$x$, drawn from the equidistribution on $B_{n}(1)$, from a random i.i.d.
sample $\mathcal{M}$ drawn from the same distribution. In practice, however,
the data or a training set are given or fixed. It is thus important to know if
the “point” linear separability property formulated in Theorem 1 persists (in
one form or another) when the test point $x$ belongs to the sample
$\mathcal{M}$ itself. In particular, the question is if the probability
$P_{M}(\mathcal{M},n)$ that each point $y\in\mathcal{M}$ is linearly separable
from $\mathcal{M}\backslash\\{y\\}$ is close to $1$ in high dimensions? If
such a property does hold then one could conclude that in high dimensions with
probability close to $1$ all points of $\mathcal{M}$ are the vertices (extreme
points) of the convex hull of $\mathcal{M}$ and none of $y\in\mathcal{M}$ is a
convex combination of other points. The fact that this is indeed the case
follows from Theorem 2

###### Theorem 2

Consider an equidistribution in a unit ball $B_{n}(1)$ in $\mathbb{R}^{n}$,
and let $\mathcal{M}$ be an i.i.d. sample from this distribution.Then

$\begin{split}&P_{M}(\mathcal{M},n)\geq\\\ &\ \
\max_{\varepsilon\in(0,1)}\left[(1-(1-\varepsilon)^{n})\left(1-(M-1)\frac{{\rho}(\varepsilon)^{n}}{2}\right)\right]^{M}.\end{split}$
(9)

Proof of Theorem 2. Let $P:\mathcal{F}\rightarrow[0,1]$ be a probability
measure and $A_{i}\in\mathcal{F}$, $i=1,\dots,M$. It is well-known that

$P(A_{1}\vee A_{2}\vee\ldots\vee A_{M})\leq\sum_{i=1}^{M}P(A_{i})$ (10)

The probability that a test point $y$ is in the $\varepsilon$-vicinity of the
boundary of $B_{n}(1)$ is $1-(1-\varepsilon)^{n}$. Fix $y\in\mathcal{M}$ and
construct spherical caps $C_{n}(\varepsilon)$ for each element in
$\mathcal{M}\backslash\\{y\\}$ as specified by (4) but with $x$ replaced by
the corresponding points from $\mathcal{M}\backslash\\{y\\}$. According to
(10) and Lemma 1, the probability that $y$ is in any of these caps is no
larger than $(M-1)\frac{\rho(\varepsilon)^{n}}{2}$. Hence the probability that
a point $y\in\mathcal{M}$ is separable from $\mathcal{M}\backslash\\{y\\}$ is
larger or equal to
$(1-(1-\varepsilon)^{n})(1-(M-1)\frac{\rho(\varepsilon)^{n}}{2})$. Given that
points of $\mathcal{M}$ are drawn independently and that there are exactly $M$
points in $\mathcal{M}$, the probability that every single point is linearly
separable from the rest satisfies (9). $\square$

###### Remark 4

Note that employing (10) one can obtain another estimate of $P_{M}$:

$P_{M}(\mathcal{M},n)\geq 1-M(1-P_{1}(\mathcal{M},n)).$ (11)

We can utilise this estimate together with (8) and estimate the maximal size
of the sample from below. Indeed, if we require that $P_{M}(\mathcal{M},n)\geq
q$ for some probability $q$, $0<q<1$, then it is sufficient that
$P_{1}(\mathcal{M},n)>p$, where $1-p=\frac{1}{M}(1-q)$. Using in (8) $|\ln
p|>1-p$, we get that $P_{M}(\mathcal{M},n)>q$ if $M\leq\tilde{M}$ for some
maximal value $\tilde{M}$, that satisfies the inequality

$\tilde{M}\geq
e^{n\ln\left(\rho(\varepsilon)^{-1}\right)}\frac{1-q}{\tilde{M}}.$

Immediately from this inequality we get the explicit exponential estimate of
the maximum of $\tilde{M}$ from below:

$\max\\{\tilde{M}\\}\geq
e^{\frac{1}{2}n\ln\left(\rho(\varepsilon)^{-1}\right)}\sqrt{1-q}$ (12)

Similarly to the example discussed in the end of the previous section, let us
evaluate the right-hand side of (9) for some fixed values of $n$ and $M$. If
$n=50$, $M=1000$, and $\varepsilon=1/5$, $\rho=3/5$ then this estimate gives:
$P_{M}(\mathcal{M},50)>0.985$.

###  3.3 Two-functional (two-neuron) separation in finite sets

Figure 3: Two-neuron separation in finite sets. Every point can be separated
by a sufficiently acute angle or highly correlated neurons: Point 1 is
separated from other points by an acute angle (dashed lines). Point 2 is
separated by a right angle (non-correlated neurons), but cannot be separated
by a linear functional (i.e. by a straight line).

So far we have provided estimates of the probabilities that a single linear
classifier or a learning machine can separate a given point from the rest of
data and showed that two disjoint weakly compact subsets of a topological
vector space can be separated by small networks of perceptrons. Let us now see
how employing small networks may improve probabilities of separation of a
point from the rest of the data in high dimensions. In particular, we will
consider the case of a two-neuron separation in which the network is a simple
cascade comprised of two perceptrons followed by a conjunction operation.

Before, however, going any further we need to somewhat clarify and adjust the
notion of separability of a point from a finite data set by a network so that
the question makes any practical and theoretical sense. Consider, for
instance, the problem of separating a test point by just two perceptrons. If
one projects the data onto a $2$d plane so that projections of the test point
and any other point from the rest of the data do not coincide then the problem
always has a solution. This is illustrated with the diagram in Fig. 3.
According to this diagram any given point from an arbitrary but finite data
set could be cut out from the rest of the data by just two lines that
intersect at a sufficiently acute angle. Thus two hyperplanes $\\{x\ |\
l_{1}(x)=\theta_{1}\\}$ and $\\{x\ |\ l_{2}(x)=\theta_{2}\\}$ whose
projections onto the $2$d plane are exactly these two lines already constitute
the two-neuron separating cascade.

The problem with this solution is that if the acute angle determined by the
inequalities $l_{1}(x)>\theta_{1}$, $l_{2}(x)>\theta_{2}$ is small then the
coefficients of the corresponding linear functionals $l_{1}(x)$ and
$l_{2}(x)$, i.e. synaptic weights of the neurons, are highly correlated. Their
Pearson correlation coefficient is close to $-1$. This implies that robustness
of such a solution is low. Small changes in the coefficients of one perceptron
could result in loss of separation. This motivates an alternative solution in
which the coefficients are uncorrelated, i.e. the angles between two
hyperplanes are right (or almost right).

Let us now analyse the problem of separation of a random i.i.d. finite sample
$\mathcal{M}$ drawn from an equidistribution in $B_{n}(1)$ from a point $x$
drawn independently from the same distribution by two non-correlated neurons.
More formally, we are interested in the probability
$\mathcal{P}_{1}(\mathcal{M},n)$ that a two-neuron cascade with uncorrelated
synaptic weights separates $x$ from $\mathcal{M}$. An estimate of this
probability is provided in the next theorem.

###### Theorem 3

Consider an equidistribution in a unit ball $B_{n}(1)$ in $\mathbb{R}^{n}$,
and let $\mathcal{M}$ be an i.i.d. sample from this distribution. Then

$\begin{split}\mathcal{P}_{1}(\mathcal{M},n)&\geq\max_{\varepsilon\in(0,1)}\
(1-(1-\varepsilon)^{n})\times\\\
&\left(1-\frac{\rho(\varepsilon)^{n}}{2}\right)^{M}\
e^{(M-n+1)\left[\frac{\frac{\rho(\varepsilon)^{n}}{2}}{1-\frac{\rho(\varepsilon)^{n}}{2}}\right]}\times\\\
&\left(1-\frac{1}{n!}\left((M-n+1)\frac{\frac{\rho(\varepsilon)^{n}}{2}}{1-\frac{\rho(\varepsilon)^{n}}{2}}\right)^{n}\right).\end{split}$
(13)

Proof. Observe that in the case of general position, a single neuron (viz.
linear functional) separates $n+1$ points with probability $1$. This means
that if no more than $n-1$ points from $\mathcal{M}$ are in the spherical cap
$C_{n}(\varepsilon)$ corresponding to the test point then the second
perceptron whose weights are orthogonal to the first one will filter out these
additional spurious $n-1$ points with probability $1$.

Let $p_{c}$ be the probability that a point from $\mathcal{M}$ falls within
the spherical cap $C_{n}(\varepsilon)$. Then the probability that only up to
$n-1$ points of $\mathcal{M}$ will be in the cap $C_{n}(\varepsilon)$ is

$\mathcal{P}(M,n)=\sum_{k=0}^{n-1}\left(\begin{array}[]{c}M\\\
k\end{array}\right)(1-p_{c})^{M-k}p_{c}^{k}$

Observe that $\mathcal{P}(M,n)$, as a function of $p_{c}$, is monotone and
non-increasing on the interval $[0,1]$, with $\mathcal{P}(M,n)=0$ at $p_{c}=1$
and $\mathcal{P}(M,n)=1$ at $p_{c}=0$. Hence taking estimate (5) into account
one can conclude:

$\mathcal{P}(M,n)\geq\sum_{k=0}^{n-1}\left(\begin{array}[]{c}M\\\
k\end{array}\right)\left(1-\frac{\rho(\varepsilon)^{n}}{2}\right)^{M-k}\left(\frac{\rho(\varepsilon)^{n}}{2}\right)^{k}.$

Noticing that

$\begin{split}&\sum_{k=0}^{n-1}\left(\begin{array}[]{c}M\\\
k\end{array}\right)\left(1-\frac{\rho(\varepsilon)^{n}}{2}\right)^{M-k}\left(\frac{\rho(\varepsilon)^{n}}{2}\right)^{k}=\\\
&\left(1-\frac{\rho(\varepsilon)^{n}}{2}\right)^{M}\sum_{k=0}^{n-1}\left(\begin{array}[]{c}M\\\
k\end{array}\right)\left(\frac{\frac{\rho(\varepsilon)^{n}}{2}}{1-\frac{\rho(\varepsilon)^{n}}{2}}\right)^{k},\end{split}$

and bounding $\left(\begin{array}[]{c}M\\\ k\end{array}\right)$ from above and
below as $\frac{(M-n+1)^{k}}{k!}\leq\left(\begin{array}[]{c}M\\\
k\end{array}\right)\leq\frac{M^{k}}{k!}$ for $0\leq k\leq n-1$ we obtain

$\mathcal{P}(\mathcal{M},n)\geq\left(1-\frac{\rho(\varepsilon)^{n}}{2}\right)^{M}\sum_{k=0}^{n-1}\frac{1}{k!}\left(\frac{(M-n+1)\frac{\rho(\varepsilon)^{n}}{2}}{1-\frac{\rho(\varepsilon)^{n}}{2}}\right)^{k}.$

Invoking Taylor’s expansion of $e^{x}$ at $x=0$ with the Lagrange reminder
term:

$e^{x}=\sum_{k=0}^{n-1}\frac{x^{k}}{k!}+\frac{x^{n}}{n!}e^{\xi},\
\xi\in[0,x],$

we can conclude that

$\sum_{k=0}^{n-1}\frac{x^{k}}{k!}\geq e^{x}\left(1-\frac{x^{n}}{n!}\right)$

for all $x\geq 0$. Hence

$\begin{split}&\mathcal{P}(\mathcal{M},n)\geq\left(1-\frac{\rho(\varepsilon)^{n}}{2}\right)^{M}e^{(M-n+1)\left[\frac{\frac{\rho(\varepsilon)^{n}}{2}}{1-\frac{\rho(\varepsilon)^{n}}{2}}\right]}\times\\\
&\left(1-\frac{1}{n!}\left((M-n+1)\frac{\frac{\rho(\varepsilon)^{n}}{2}}{1-\frac{\rho(\varepsilon)^{n}}{2}}\right)^{n}\right).\end{split}$

Finally, given that the probability that the test point $x$ is in the
$\varepsilon$-vicinity of the boundary of $B_{n}(1)$ is at least
$(1-(1-\varepsilon)^{n})$ and that $x$ is independently selected form the same
eqidistribution as the set $\mathcal{M}$, we obtain
$\mathcal{P}_{1}(\mathcal{M},n)\geq(1-(1-\varepsilon)^{n})\mathcal{P}(\mathcal{M},n)$.
This in turn implies that (13) holds. $\square$

At the first glance estimate (13) looks more complicated as compared to e.g.
(2). Yet, it differs from the latter by mere two factors. The first factor

$\left(1-\frac{1}{n!}\left((M-n+1)\frac{\frac{\rho(\varepsilon)^{n}}{2}}{1-\frac{\rho(\varepsilon)^{n}}{2}}\right)^{n}\right)$

is close to $1$ for $(M-n+1)\frac{\rho(\varepsilon)^{n}}{2}<1$ and $n$
sufficiently large. The second factor:

$e^{(M-n+1)\left[\frac{\frac{\rho(\varepsilon)^{n}}{2}}{1-\frac{\rho(\varepsilon)^{n}}{2}}\right]},$

is more important. It compensates for the decay of the probability of
separation due to the term $(1-\frac{\rho(\varepsilon)^{n}}{2})^{M}$ keeping
the rhs of (13) close to $1$ over large interval of values of $M$. The effect
is illustrated with Fig. 4. Observe that the probability corresponding to the
two-neuron cascade remains in a close vicinity of $1$, and the probability of
one-neuron separation gradually decays with $M$.

Figure 4: Illustration to Theorem 3. Blue line shows an estimate of the rhs of
(13) as a function of $M$ for $n=30$. Red line depicts an estimate of the rhs
of (2) as a function of $M$ for the same values of $n$.

###### Remark 5

Comparing performance of single vs two-neuron separability in terms of the
probabilities $\mathcal{P}_{1}(\mathcal{M},n)$ involves taking the maximum of

$\mathcal{P}_{1}(\mathcal{M},n,\varepsilon)=(1-(1-\varepsilon)^{n})\left(1-\frac{{\rho}(\varepsilon)^{n}}{2}\right)^{M}$
(14)

and

$\begin{split}\mathcal{P}_{1}(\mathcal{M},n,\varepsilon)=&(1-(1-\varepsilon)^{n})\times\\\
&\left(1-\frac{\rho(\varepsilon)^{n}}{2}\right)^{M}\
e^{(M-n+1)\left[\frac{\frac{\rho(\varepsilon)^{n}}{2}}{1-\frac{\rho(\varepsilon)^{n}}{2}}\right]}\times\\\
&\left(1-\frac{1}{n!}\left((M-n+1)\frac{\frac{\rho(\varepsilon)^{n}}{2}}{1-\frac{\rho(\varepsilon)^{n}}{2}}\right)^{n}\right)\end{split}$
(15)

with respect to $\varepsilon$ over $(0,1)$. In some situations, when the
testing point is already given, the probabilities
$\mathcal{P}_{1}(\mathcal{M},n)$ are no longer relevant since the value of
$\varepsilon$ corresponding to the testing point is fixed. In this cases one
needs to compare $\mathcal{P}_{1}(\mathcal{M},n,\varepsilon)$ defined by (14)
and (15) instead. Performance of the corresponding separation schemes are
illustrated with Fig. 5. Notice that the two-neuron cascade significantly
outperforms the single neuron one over a large interval of values of $M$.

Figure 5: Illustration to Remark 5. Blue line shows an estimate of the rhs of
(15) as a function of $M$ at $\varepsilon=1/5$, $\rho(\varepsilon)=3/5$, and
$n=30$. Red line depicts an estimate of the rhs of (14) as a function of $M$
for the same values of $\varepsilon$, $\rho(\varepsilon)$, and $n$.

###### Remark 6

The probability $\mathcal{P}_{M}(\mathcal{M},n)$ that each point from
$\mathcal{M}$ can be separated from other points by two uncorrelated neurons
can be estimated like in Remark 4: $\mathcal{P}_{M}(\mathcal{M},n)\geq
1-M(1-\mathcal{P}_{1}(\mathcal{M},n))$.

###  3.4 Separation by small neural network

In the previous sections we analysed and discussed linear separability of
single elements of $\mathcal{M}$. As far as the problem of a legacy AI system
corrector is concerned, however, it is not difficult to envision a need to
implement more complicated dependencies, including logical predicates or
involving more complicated data geometry. Below we provide some notions and
results enabling us to deal with these issues.

Consider a topological real vector space $L$. For every continuous linear
functional $l$ on $L$ and a real number $\theta$ we define an open elementary
neural predicate $l(x)>\theta$ and a closed elementary predicate
$l(x)\geq\theta$ (‘open’ and ‘closed’ are sets defined by predicates). The
elementary neural predicate is true (=1) or false (=0) depending on whether
the correspondent inequality holds. The negation of an open elementary neural
predicate is a closed elementary neural predicate (and converse). A compound
predicate is a Boolean expression composed of elementary predicates.

###### Definition 2

A set $X\subset L$ is $k$-neuron separable from a set $Y\subset L$ if there
exist $k$ elementary neural predicates $P_{1},\ldots,P_{k}$ and a composed of
them compound predicate, a Boolean function
$B(x)=B(P_{1}(x),\ldots,P_{k}(x))$, such that $B(x)=$true for $x\in X$ and
$B(x)=$false for $x\in Y$.

If $X$ consist of one point then the compound predicate in the definition of
$k$-neuron separability can be selected in a form of conjunction of $k$
elementary predicates $B(x)=P_{1}(x)\&\ldots\&P_{k}(x))$:

###### Proposition 1

Let $X=\\{z\\}$ and $X$ is $k$-neuron separable from $Y\subset L$. Then there
exist $k$ elementary neural predicates $P_{1},\ldots,P_{k}$ such that the
Boolean function $B(x)=P_{1}(x)\&\ldots\&P_{k}(x)$ is true on $X$ and false on
Y.

Proof. Assume that there exist $k$ elementary neural predicates
$Q_{1},\ldots,Q_{k}$ and a Boolean function $B(Q_{1},\ldots,Q_{k})$ such that
$B$ is true on $x$ and false on $Y$. Represent $B$ in a disjunctive normal
form as a disjunction of conjunctive clauses:

$B=C_{1}\lor C_{2}\lor\ldots\lor C_{N},$

where each $C_{i}$ has a form $C_{i}=R_{1}\&R_{2}\&\ldots\&R_{k}$ and
$R_{i}=Q_{i}$ or $R_{i}=\neg Q_{i}$.

At least one conjunctive clause $C_{i}(x)$ is true because $B(x)=$true. On the
other hand, $C_{i}(y)=$false for all $y\in Y$ and $i=1,\ldots,N$ because their
disjunction $B(y)$=false. Let $C_{i}(x)$=true. Then $C_{i}$ separates $x$ from
$Y$ and other conjunctive clauses are not necessary in $B$. We can take
$B=C_{i}=R_{1}\&R_{2}\&\ldots\&R_{k}$, where $R_{i}=Q_{i}$ or $R_{i}=\neg
Q_{i}$. Finally, we take $P_{i}=R_{i}$. $\square$

In contrast to Hahn-Banach theorems of linear separation, the $k$-neuron
separation of sets does not require any sort of convexity. Weak compactness is
sufficient. Assume that continuous linear functionals separate points in $L$.
Let $Y\subset L$ be a weakly compact set [40] and $x\notin Y$.

###### Proposition 2

$x$ is $k$-neuron separable from $Y$ for some $k$.

Proof. For each $y\in Y$ there exists a continuous linear functional $l_{y}$
on $L$ such that $l_{y}(x)=l_{y}(y)+2$ because continuous linear functionals
separate points in $L$. Inequality $l_{y}(z)<l_{y}(x)-1$ defines an open half-
space $L_{y}^{<}=\\{z\ |\ l_{y}(z)<l_{y}(y)+1\\}$, and $y\in L_{y}^{<}$. The
collection of sets $\\{L_{y}^{<}\ |\ y\in Y\\}$ forms an open cover of $Y$.
Each set $L_{y}^{<}$ is open in weak topology and $Y$ is weakly compact, hence
there exists a final covering of $Y$ by sets $L_{y}^{<}$:

$Y\subset\bigcup_{i=1,\ldots,k}L_{y_{i}}^{<}$

for some finite subset $\\{y_{1},\ldots,y_{k}\\}\subset Y$. The inequality
$l_{y_{i}}(x)>l_{y_{i}}(y_{i})+\alpha$ holds for all $i=1,\ldots,k$ and
$\alpha<2$. Let us select $1<\alpha<2$ and take elementary neural predicates
$P_{i}^{\alpha}(x)=(l_{y_{i}}(x)>l_{y_{i}}(y_{i})+\alpha)$. The conjunction
$P_{1}^{\alpha}\&\ldots\&P_{k}^{\alpha}$ is true on $x$. Each point $y\in Y$
belongs to $L_{y_{i}}^{<}$ for at least one $i=1,\ldots,k$. For this $i$,
$P_{i}^{\alpha}(y)$=false. Therefore, $P_{1}^{\alpha}\&\ldots\&P_{k}^{\alpha}$
is false on $Y$. Hence, this conjunction separates $x$ from $Y$. $\square$

###### Remark 7

Note that we can select two numbers $1<\beta<\alpha<2$. Both
$P_{1}^{\alpha}\&\ldots\&P_{k}^{\alpha}$ and
$P_{1}^{\beta}\&\ldots\&P_{k}^{\beta}$ separate $x$ from $Y$.

The theorem about $k$-neuron separation of weakly compact sets follows from
Proposition 2. Let $X$ and $Y$ be disjoint and weakly compact subsets of $L$,
and continuous linear functionals separate points in $L$. Then the following
statement holds

###### Theorem 4

$X$ is $k$-neuron separable from $Y$ for some $k$.

Proof. For each $x\in X$ apply the construction from the proof of Proposition
2 and construct the conjunction
$B_{x}=P_{1}^{\alpha}\&\ldots\&P_{k}^{\alpha}$, which separates $x$ from $Y$.
Every set of the form $F_{x}=\\{z\in L\ |\ B_{x}(z)=\mbox{true}\\}$ is
intersection of a finite number of open half-spaces and, therefore, is open in
weak topology. The collection of sets $\\{F_{x}\ |\ x\in X\\}$ covers $X$.
There exist a final cover of $X$ by sets $F_{x}$:

$X\subset\bigcup_{j=1,\ldots,g}F_{x_{j}}$

for some finite set $\\{x_{1},\ldots,x_{l}\\}\subset X$. The composite
predicate that separates $X$ from $Y$ can be chosen in the form
$B(z)=B_{x_{1}}(z)\lor\ldots\lor B_{x_{l}}(z)$. $\square$

##  4 Discussion

The nature of phenomenon described is universal and does not depend on many
details of the distribution. In some sense, it should be just essentially
high-dimensional. Due to space limitations cannot review all these
generalizations here but will nevertheless provide several examples.

###  4.1 Equidistribution in an ellipsoid

Let points in the set $\mathcal{M}$ be selected by independent trials taken
from the equidistribution in a $n$-dimensional ellipsoid. Without loss of
generality, we present this ellipsoid in the orthonormal eigenbasis:

$E_{n}=\\{x\in\mathbb{R}^{n}|\sum_{i=1}^{n}\frac{x_{i}^{2}}{c_{i}^{2}}\leq
1\\},$ (16)

where $rc_{i}$ are the semi-principal axes. The linear transformation

$(x_{1},\ldots,x_{n})\mapsto\left(\frac{x_{1}}{c_{1}},\ldots,\frac{x_{n}}{c_{n}}\right)$

transforms the ellipsoid into the unit ball. The volume of every set in the
new coordinates scales with the factor $1/\prod_{i}c_{i}$. Therefore the ratio
of two volumes does not change, and the equidistribution in the ellipsoid is
transformed into the equidistribution in the unit ball. Hyperplanes are
transformed into hyperplanes and the property of linear separability is not
affected by a nonsingular linear transformation. Thus, the following
corollaries from Theorems 1, 2 hold.

###### Corollary 1

Let $\mathcal{M}$ be formed by a finite number of i.i.d. trials taken from the
equidistribution in a $n$-dimensional ellipsoid $E_{n}$ (16), and let $x$ be a
test point drawn independently from the same distribution. Then $x$ can be
separated from $\mathcal{M}$ by a linear functional with probability
$P_{1}(\mathcal{M},n)$:

$P_{1}(\mathcal{M},n)\geq(1-(1-\varepsilon)^{n})\left(1-\frac{\rho(\varepsilon)^{n}}{2}\right)^{M}.$

###### Corollary 2

Let $\mathcal{M}$ be formed by a finite number of i.i.d. trials taken from the
equidistribution in a $n$-dimensional ellipsoid $E_{n}$. With probability

$P_{M}(\mathcal{M},n)\geq\left[(1-(1-\varepsilon)^{n})\left(1-(M-1)\frac{\rho(\varepsilon)^{n}}{2}\right)\right]^{M}$

each point $y$ from the random set $\mathcal{M}$ can be separated from
$\mathcal{M}\backslash\\{y\\}$ by a linear functional.

###  4.2 Equidistributions in a cube and normal distributions

It is well known that, for $n$ sufficiently large, samples drawn from an
$n$-dimensional normal distribution [30] concentrate near a corresponding
$n$-sphere. Similarly, samples generated from an equidistribution in an
$n$-dimensional cube concentrate near a corresponding $n$-sphere too. In this
respect, one might expect that the estimates derived in Theorems 1-3 hold for
these distributions too, asymptotically in $n$. Our numerical experiments
below illustrate that this is indeed the case.

The experiments were as follows. For each distribution (normal distribution in
$\mathbb{R}^{n}$ and equidistribution in the $n$-cube $[-1,1]^{n}$) and a
given $n$ an i.i.d. sample $\mathcal{M}$ of $M=10^{4}$ vectors was drawn. For
each vector $y$ in this sample we constructed the functional $l_{y}(x)=\langle
y,x\rangle-\|y\|^{2}$ (cf. the proof of Lemma 1). For each $y\in\mathcal{M}$
and $x\in\mathcal{M}$, $x\neq y$ the sign of $l_{y}(x)$ was evaluated, and the
total number $N$ of instances when $l_{y}(x)<0$ for all $x\in\mathcal{M}$,
$x\neq y$ was calculated. The latter number is a lower bound estimate of the
number of points in $\mathcal{M}$ that are linearly separable from the rest in
the sample. This was followed by deriving the values of the success
frequencies, $F_{1}(\mathcal{M},n)=N/(M-1)$. For each $n$ the experiment was
repeated $50$ times. Outcomes of these experiments are presented in Fig. 6

Dimension, $n$ | $2$ | $5$ | $10$ | $20$ | $30$
---|---|---|---|---|---
Ball |  |  |  |  |
(theoretical estimate) | $3.7\cdot 10^{-5}$ | $0.0364$ | $0.4096$ | $0.9455$ | $0.9975$
Cube |  |  |  |  |
min: | $4\cdot 10^{-4}$ | $0.0089$ | $0.2580$ | $0.9408$ | $0.9986$
median: | $4\cdot 10^{-4}$ | $0.0110$ | $0.2737$ | $0.9469$ | $0.9992$
max: | $5\cdot 10^{-4}$ | $0.0137$ | $0.2847$ | $0.9511$ | $1.0$
Gaussian |  |  |  |  |
min: | $5\cdot 10^{-4}$ | $0.0122$ | $0.1403$ | $0.7559$ | $0.9792$
median: | $10\cdot 10^{-4}$ | $0.0153$ | $0.1568$ | $0.7698$ | $0.9817$
max: | $0.0014$ | $0.0183$ | $0.1778$ | $0.7836$ | $0.9848$

Figure 6: Numerical and theoretical estimates of $P_{1}(\mathcal{M},n)$ for
various distributions and $n$. Top panel, top row: theoretical estimate of
$P_{1}(\mathcal{M},n)$ for equidistributions in $n$-balls derived in
accordance with (2) in the statement of Theorem 1. Left panel, rows $2,3$:
numerical estimates $F_{1}(\mathcal{M},n)$ of $P_{1}(\mathcal{M},n)$ for both
normal and equidistribution in an $n$-cube for various values of $n$. Bottom
panel: solid circles show the values of theoretical estimates
$P_{1}(\mathcal{M},n)$, triangles show empirical means of
$F_{1}(\mathcal{M},n)$ for the samples drawn from equidistribution in the cube
$[-1,1]^{n}$, and squares correspond to empirical means of
$F_{1}(\mathcal{M},n)$ for the samples drown from the Gaussian (normal)
distribution. Whiskers in the plots indicate maximal and minimal values in of
$F_{1}(\mathcal{M},n)$ in each group of experiments.

As we can see from Fig. 6, despite that the samples are drawn from different
distributions, for $n>30$ these differences do not significantly affect point
separability properties. This is due to pronounced influence of measure
concentration in these dimensions.

###  4.3 One trial non-iterative learning

Our basic model of linear separation of a given query point $y\in\mathcal{M}$
from any other $x\in\mathcal{M}$, $x\neq y$ was

$l_{y}(x)=\left\langle\frac{y}{\|y\|},x\right\rangle-\|y\|<0\mbox{ for
}x\in\mathcal{M}\smallsetminus\\{y\\}$ (17)

Deriving these functionals is a genuine one-shot procedure and does not
require iterative learning. The construction, however, assumes that the
distribution from which the sample $\mathcal{M}$ is drawn is close in some
sense to an equidistribution in the unit ball $B_{n}(1)$.

In more general cases, e.g. when sampling from the equidistribution in an
ellipsoid, the functionals $l_{y}(x)$ can be replaced with Fisher linear
discriminants:

$l_{y}(x)=\left\langle\frac{w(y,\mathcal{M})}{\|w(y,\mathcal{M})\|},x\right\rangle-c,$
(18)

where

$w(y,\mathcal{M})=\Sigma^{-1}\left(y-{\frac{1}{M-1}}\sum_{x\neq y}x\right),$

$\Sigma$ is the non-singular covariance matrix of the sample $\mathcal{M}$,
and $c$ is a parameter. The value of $c$ could be chosen as
$c=\left\langle\frac{w(y,\mathcal{M})}{\|w(y,\mathcal{M})\|},y\right\rangle$.
The procedure for generating separating functionals $l_{y}(x)$ remains non-
iterative, but it does require knowledge of the covariance matrix $\Sigma$.

Note that if $\mathcal{M}$ is centered at $0$ then ${\frac{1}{M-1}}\sum_{x\neq
y}x\simeq 0$, and (18) reduces to (17) after the corresponding Mahalanobis
transformation: $x\mapsto\Sigma^{-1/2}x$. If a transformation
$x\mapsto\Sigma^{-1/2}x$ transforms the ellipsoid from which the sample
$\mathcal{M}$ is drawn into the unit ball then (18) becomes equivalent to
(17), and separation properties of Fisher discriminants (18) follow in the
same way as stated in Corollaries 1 and 2.

In addition, or as an alternative, whitening or decorrelation transformations
could be applied to $\mathcal{M}$ too. In case the covariance matrix $\Sigma$
is singular or ill-conditioned, projecting the sample $\mathcal{M}$ onto
relevant Principal Components may be required.

###  4.4 Extreme selectivity of biological neurons

High separation capabilities of linear functionals (Theorems 1–3, Remark 3)
may shed light on the puzzle associated with the so-called Grandmother [37],
[45] or Concept cells [36] in neuroscience. In the center of this puzzle is
the wealth of empirical evidence that some neurons in human brain respond
unexpectedly selective to particular persons or objects. Not only brain is
able to respond selectively to “rare” individual stimuli but also such
selectivity can be acquired very rapidly from limited number of experiences
[24]. The question is: Why small ensembles of neurons may deliver such a
sophisticated functionality reliably? As follows from Theorems 1–3, observed
extreme selectivity of neuronal responses may be attributed to the very nature
of basic physiological mechanisms in neurons involving calculation of linear
functionals (1).

##  5 Examples

Let us now illustrate our theoretical results with examples of correcting
legacy AI systems by linear functionals (1) and their combinations. As a
legacy AI system we selected a Convolutional Neural Network trained to detect
objects in images. Our choice of the legacy system was motivated by that these
networks demonstrate remarkably strong performance on benchmarks and,
reportedly, may outperform humans on popular classification tasks such as
ILSVRC [21]. Despite this, as has been mentioned earlier, even for the most
current state of the art CNNs false positives are not uncommon. Therefore, we
investigate in the following experiments if it is possible to take one of
these cutting edge networks and train a one-neuron filter to eliminate all the
false positives produced. We will also look at what effect, if any, this
filter will have on true positive numbers.

###  5.1 Training the CNN

For these experiments we trained the VGG-11 convolutional network to be our
classifier [41]. Instead of the 1000 classes of Imagenet we trained it to
perform the simpler task of identifying just one object class: pedestrians.

The VGG network was chosen for both its simple homogeneous architecture as
well as its proven classification ability at recent Imagenet competitions
[38]. We chose to train the VGG-11 $A$ over the deeper $16$ and $19$ layer VGG
networks due to hardware and time constraints.

Datasets: In order to train the network a set of $114,000$ positive pedestrian
and $375,000$ negative non-pedestrian RGB images, re-sized to $128\times 128$,
were collected and used as a training set. Our testing set comprised of
further $10,000$ positives and $10,000$ negatives. The training and testing
sets had no common elements.

The training procedure and choice of hyper parameters for the network follows
largely from [41]. but with some small adjustments to take into account the
reduced number of classes being detected. We set momentum to $0.9$ and used a
mini batch size of $32$.

Our initial learning rate was set to $0.00125$ and this rate was reduced by a
factor of $10$ after $25$ epochs and again after $50$ epochs. Dropout
regularisation was used for the first two fully connected layers with a ratio
of $0.5$. To initialise weights we used the Xavier initialisation [14] which
we found helped training to converge faster. Finally, we stopped the learning
after $75$ epochs in order to avoid overfitting.

Training the network was done using the publicly available Caffe deep learning
framework [26]. For the purposes of evaluating performance of the network, our
null-hypothesis was that no pedestrians are present in the scanning window.
True positive hence is an image crop (formed by the scanning window)
containing a pedestrian and for which the CNN reports such a presence, false
positive is an image crop for which the CNN reports presence of a pedestrian
whilst none are present. True negatives and false negatives are defined
accordingly. On the training set, the VGG-11 convolutional network showed
$100\%$ correct classification performance: the rates of true positives and
true negatives were all equal to one.

###  5.2 Training one-neuron legacy AI system correctors

####  5.2.1 Error detection and selection of features

First, a multi-scale sliding window approach is used on each video frame to
provide proposals that can be run through the trained CNN. These proposals are
re-sized to $128\times 128$ and passed through the CNN for classification,
non-maximum suppression is then applied to the positive proposals. Bounding
boxes are drawn and we compared results to a ground truth so we can identify
false positives.

For each positive and false positive (and its respective set of proposals
before non-maximum suppression) we extracted the second to last fully
connected layer from the CNN. These extracted feature vectors have dimension
$4096$. We used these feature vectors to construct one-neuron correctors
filtering out the false positives that have been manually identified. These we
will refer to in the text as ’trash models’.

####  5.2.2 Spherical cap model, (17)

The first and the most simple trash model filter that we constructed was the
spherical cap model as specified by (17). In this model, after centering the
training data, the query points $y$, adjusted for the mean of the training
data, were the actual false positives detected.

####  5.2.3 Fisher cap model, (18)

Our second model type was the Fisher cap model (18). The original model
involves the covariance matrix $\Sigma$ corresponding to $4096$ dimensional
feature vectors of positives in the training set. Examination of this matrix
revealed that the matrix is extremely ill-conditioned and hence (18) cannot be
used unless the problem is regularized. To overcome the singularity issue the
PCA-based dimensionality reduction was employed. To estimate the number of
relevant principal components needed we performed the broken stick [31] and
the Kaiser-Guttman [25] tests. The broken stick criterion returned the value
of $13$, and the Kaiser rule returned $45$. The distribution of the
eigenvalues of $\Sigma$ is shown in Fig. 7. We used these data to select lower
and upper bound estimates of the number of feasible principal components
needed. For the purposes of our experiments these were chosen to be $50$ and
$2000$, respectively.

Figure 7: Log-log plot of the eigenvalues $\lambda_{i}(\Sigma)$ of the
training data covariance matrix $\Sigma$.

####  5.2.4 Support Vector Machine (SVM)

As the ultimate linear separability benchmark, classical linear SVM model has
been used. It was trained using liblinear [10] on our two sets of normalised
CNN feature vectors.

Something important to note: we found that as the number of false positives
being trained on increased it was necessary to also increase the value of the
C parameter to maintain a perfect separation of the training points. This
perfect separation was required to minimise the loss of true positives while
simultaneously removing as many of the false positives as possible.

###  5.3 Implementation

At test time trained linear corrector models (trash model filters) were placed
at the end of our detection pipeline, each one at a time. For any proposal
given a positive score by our CNN we again extract the second to last fully
connected layer. The CNN feature vectors from these positive proposals are
then run through the trash model filter.

Any detection that then gives a positive score from the linear trash model is
consequently removed by turning its detection score negative.

###  5.4 Results

Test videos: For our experiments we used the original training and testing
sets as well as three different videos (not used in the training set
generation) to test trash model creation and its effectiveness at removing
false positives. The first sequence is the NOTTINGHAM video [6] that we
created ourselves from the streets of Nottingham consisting of $435$ frames
taken with an action camera. The second video is the INRIA test set consisting
of $288$ frames [8] , the third is the LINTHESCHER sequence produced by ETHZ
consiting of $1208$ frames [9]. For each test video a new trash model was
trained.

####  5.4.1 Spherical cap model, (17)

To test performance of this naïve model we run the trained CNN classifier
through the testing set and identified false positives. From the set of false
positives, candidates were selected at random and correcting trash model
filters (17) constructed. Performance of the resulting systems was then
evaluated on the sets comprising of $114,000$ positives (from the training
set) and $1$ negative (the false positive identified). These tests returned
true negative rates equal to $1$, as expected, and $0.978884$ true positive
rate averaged over $25$ experiments. This is not surprising given that the
model does not account for any statistical properties of the data apart from
the assumption that the sample is taken from an equidistribution in
$B_{n}(1)$. The latter assumption, as is particularly evident from Fig. 7,
does not hold either. Equidistribution in an ellipsoid (or the
multidimensional normal distribution, which is essentially the same) is a much
better model of the data, and hence one would expect that Fisher discriminants
(18) would perform better. We shall see now that this is indeed the case.

####  5.4.2 Fisher cap model, (18)

The Fisher discriminant is essentially the functional we use in theoretical
estimate for distributions in ellipsoid. It is the same spherical cup, albeit
after the whitening applied. The Fisher cap models were first constructed and
tested on the data used to train and test the original CNN classifier. We
started with the data projected onto the first $50$ principal components. The
$25$ false positive candidates, taken from the testing set, were chosen at
random as described in the previous section. They modeled single mistakes of
the legacy classifier. The models were then prepared using the covariance
matrices of the original training data set. On the data sets comprising of
$114,000$ positives from the training set and $1$ negative corresponding to
the single randomly chosen identified false positive all models delivered true
negative and true positive rates equal to $1$. The result persisted when the
number of principal components used was increased to $2000$.

In the next set of experiments we asked if a single neuron with the weights
given by the Fisher discriminant formula would be capable of filtering out
more than just one false positive candidate. For this purpose we randomly
selected several samples $\mathcal{Y}$ of $25$ false positive candidates from
the training data set and constructed models (18) aiming at separating all
$25$ false positives in each sample from $114,000$ true positives in the data
set. The weights $w$ of model (18) depended on the sets of false positives
$\mathcal{FP}$ and the set $\mathcal{TP}$ of true positives as follows:

$\begin{split}w(\mathcal{FP},\mathcal{TP})=&\left(\Sigma_{\mathcal{TP}}+\Sigma_{\mathcal{FP}}\right)^{-1}\left(\frac{1}{|\mathcal{FP}|}\sum_{y\in\mathcal{FP}}y-\frac{1}{|\mathcal{TP}|}\sum_{x\in\mathcal{TP}}x\right)\end{split}$

where $\Sigma_{\mathcal{TP}}$, $\Sigma_{\mathcal{FP}}$ are the covariance
matrices of true and false positives, respectively, and the value of parameter
$c$ was chosen as

$c=\min_{y\in\mathcal{FP}}\left\langle\frac{w(\mathcal{FP},\mathcal{TP})}{\|w(\mathcal{FP},\mathcal{TP})\|},y\right\rangle.$

We did not observe perfect separation if only the first $50$ principal
components were used to construct the models. Perfect separation, however, was
persistently observed when the number of first principle components being used
exceeded $87$. For data projected on more than the first $87$ principal
components one neuron with weights selected by the Fisher linear discriminant
formula corrected $25$ errors without doing any damage to classification
capabilities (original skills) of the legacy AI system on the training set.

In the third group of tests we built single neuron trash models (18) on
varying numbers of false positives from the NOTTINGHAM video. The true
positives (totalling to $2896$) and the false positives ($189$) in this video
differed from those in the training set. To construct the covariance matrices
$\Sigma$, the original training data set was projected onto the first $2000$
principal components. Typical performance of the legacy CNN system with the
corrector is shown in Fig. 8.

Figure 8: Performance of one-neuron corrector built using Fisher cap model
(18). Left panel: the number of false positives removed as a function of the
number of false positives the model was built on. Stars indicate the number of
false positives used for building the model. Squares correspond to the number
of false positives removed by the model. Right panel: the number of true
positives removed as a function of the number of false positives removed. The
actual measurements are shown as squares. Solid line is the least-square fit
of a quadratic.

As we can see from this figure, single false positives are dealt with
successfully without any detrimental affects on the true positive rates.
Increasing the number of false positives that the single neuron trash model is
to cope with resulted in gradual deterioration of the true positive rates. One
may expect that, according to Theorem 3, increasing of the number of neurons
in trash models could significantly improve the power of correctors.
Conducting these experiments, however, was outside of the main focus of this
example.

####  5.4.3 SVM model

The Spherical cap and the Fisher cap models are by no means the ultimate tests
of separability. Their main advantage is that they are purely non-iterative.
To test extremal performance of linear functionals more sophisticated methods
for their construction, such as e.g. SVMs, are needed. To assess this
capability we trained SVM trash models on varying numbers of false positives
from the NOTTINGHAM video, as a benchmark. Results are plotted in Fig. 9
below. The true positives were taken from the CNN training data set.

Figure 9: Performance of one-neuron corrector constructed using the linear SVM
model. Left panel: the number of false positives removed as a function of the
number of false positives the model was trained on. Stars indicate the number
of false positives used for training the model. Circles correspond to the
number of false positives removed by the model Right panel: the number of true
positives removed as a function of the number of false positives removed. The
actual measurements are presented as circles. Solid line is the least-square
fit of a quadratic.

The SVM trash model successfully removes $13$ false positives without
affecting the true positives rate. Its performance deteriorates at a much
slower rate than for the Fisher cap model. This, however, is balanced by the
fact that the Fisher cap model removed significantly more false positives than
it was trained on.

Similar level of performance was observed for other testing videos, including
INRIA and ETH LINTHESCHER videos. The results are provided in Tabes 1 and 2
below.

Table 1: CNN performance on the INRIA video with and without trash model filtering |  Without trash model |  With SVM trash model
---|---|---
True positives |  490 |  489
False positives |  31 |  0
Table 2: CNN performance on the ETH LINTHESCHER video with and without trash model filtering |  Without trash model |  With SVM trash model
---|---|---
True positives |  4288 |  4170
False positives |  9 |  0

###  5.5 Results summary

From the results in Figures 8, 9 and Tables 1, 2 we have demonstrated that it
is possible to train a simple single-neuron corrector capable of rectifying
mistakes of legacy AI systems. As can be seen from Figures 8, 9 this filtering
of false positives comes at a varying cost to the true positive detections. As
we increase the number of false positives that we train the trash model on we
see the number of true positives removed starts to rise as well. Our tests
illustrate, albeit empirically, that the separation theorems stated in this
paper are viable in applications. What is particularly remarkable is that we
were able to remove more than $10$ false positives at no cost to true positive
detections in the NOTTINGHAM video by the use of a single linear function.

From practical perspectives, the main operational domain of the correctors is
likely to be near the origin along the ‘removed false positives’ axis. In this
domain generalization capacities of Fisher distriminant are significantly
higher than those of the SVMs. As can be seen from Fig. 8, Fig. 9, left
panels, Fisher discriminants are also efficient in ’cutting off’ significantly
more mistakes than they have been trained on. In this operational domain near
the origin, they do so without doing much damage to the legacy AI system. SVMs
perform better for larger values of false positives. This is not surprising as
their training is iterative and accounts for much more information about the
data than simply sample covariance and mean.

##  6 Conclusion

Stochastic separation theorems allowed us to create non-iterative one-trial
correctors for legacy AI systems. These correctors were tested on benchmarks
and on real-life industrial data and outperformed the theoretical estimates.
In these experiments, already one-neuron correctors fixed more than ten
independent mistakes without doing any damage to existing skills of the legacy
AI system. Not only these experiments illustrated application of the approach,
but also they revealed importance of whitening and regularization for
successful correction. In order to reduce the impact on true positive
detections a two neuron cascade classifier could be trained as a corrector.
Further elaboration of the theory of non-iterative one-trial correctors for
legacy AI systems and testing them in applications will be the subject of our
future work.

The stochastic separation theorems are proven for large sets of randomly drawn
i.i.d. data in high dimensional space. Each point can be separated from the
rest of the sample by a hyperplane, with high probability. In a convex hull of
a random finite set, all points of the set are vertexes (with high
probability). These results are valid even for exponentially large samples.
The estimates of the separation probability and allowable sample size age
given by Eqs. (2), (8), (9), and (12). It is not much surprising that the
separation by small ensembles of uncorrelated neurons is much more efficient
than the separation by single neurons (Figure 4).

We showed that high dimensionality of data can play a major and positive role
in various machine learning and data analysis tasks, including problems of
separation, filtration, and selection.

In contrast to naive intuition suggesting that high dimensionality of data
more often than not brings in additional complexity and uncertainty, our
findings contribute to the idea that high data dimensionality may constitute a
valuable blessing too. The term ‘blessing of dimensionality’ was used in [1]
to describe separation by Gaussians mixture models or spheres, which are
equivalent in high dimension. This blessing, formulated here in the form of
several stochastic separation theorems, offers several new insights for big
data analysis. This is a part of measure concentration phenomena [18, 19, 32]
which form the background of classical statistical physics (Gibbs theorem
about equivalence of microcanonic and canonic ensembles [13]) and asymptotic
theorems in probability [30]. In machine learning, these phenomena are in the
background of the random projection methods [27], learning large Gaussian
mixtures [1], and various randomized approaches to learning [46], [39] and
bring light in the theory of approximation by random bases [16]. It is highly
probable that the recently described manifold disentanglement effects [5] are
universal in essentially high dimensional data analysis and relate to
essential multidimensionality rather than to specific deep learning
algorithms. This may be the manifestation of concentration near the parts of
the boundary with the lowest curvature.

The plan of further technical development of the theory and methods presented
in the paper is clear:

  * 1.

To prove the separation theorems in much higher generality: for sufficiently
regular essentially multidimensional probability distribution and a random set
of weakly dependent (or independent) samples every point is linearly separated
from the rest of the set with high probability.

  * 2.

To find explicit estimation of the allowed sample size in general case.

  * 3.

To elaborate the theorems and estimates in high generality for separation by
small neural networks.

  * 4.

To develop and test generators of correctors for various standard AI tasks.

But the most inspiring consequence of the measure concentration phenomena is
the paradigm shift. It is a common point of view that the complex learning
systems should produce complex knowledge and skills. On contrary, it seems to
be possible that the main function of many learning system, both technical and
biological, in addition to production of simple skills, is a special
preprocessing. They transform the input flux (‘reality’) into essentially
multidimensional and quasi-random distribution of signals and images plus, may
be, some simple low dimensional and more regular signal. After such a
transformation, ensembles of non-interacting or weakly interacting small
neural networks (‘correctors’ of simple skills) can solve complicated
problems.

## References

  * Anderson et al. [2014] J. Anderson, M. Belkin, N. Goyal, L. Rademacher, J. Voss, The More, the Merrier: the Blessing of Dimensionality for Learning Large Gaussian Mixtures, Journal of Machine Learning Research: Workshop and Conference Proceedings 35 (2014) 1–30.
  * Ball [1997] K. Ball, An elementary introduction to modern convex geometry, Flavors of geometry 31 (1997) 1–58.
  * Bennett [1995] K. Bennett, Legacy systems: coping with success, IEEE Software 12 (1) (1995) 19–23.
  * Bisbal et al. [1999] J. Bisbal, D. Lawless, B. Wu, J. Grimson, Legacy information systems: issues and directions, IEEE Software 16 (5) (1999) 103–111.
  * Brahma et al. [2016] P. P. Brahma, D. Wu, Y. She, Why deep learning works: a manifold disentanglement perspective, IEEE Transactions On Neural Networks And Learning Systems 27 (10) (2016) 1997–2008.
  * Burton [2016] R. Burton, NOTTINGHAM video, URL https://youtu.be/SJbhOJQCSuQ, a test video for pedestrians detection taken from the streets of Nottingham by an action camera, 2016.
  * Chen et al. [2015] T. Chen, M. Li, Y. Li, M. Lin, N. Wang, M. Wang, T. Xiao, B. Xu, C. Zhang, Z. Zhang, MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems, https://github.com/dmlc/mxnet, 2015.
  * Dalal and Triggs [2005] N. Dalal, B. Triggs, Histograms Of Oriented Gradients For Human Detection, in: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 886–893, 2005.
  * Ess et al. [2008] A. Ess, B. Leibe, K. Schindler, L. van Gool, A Mobile Vision System for Robust Multi-Person Tracking, in: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 1–8, DOI: 10.1109/CVPR.2008.4587581, 2008.
  * Fan et al. [2008] R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, C.-J. Lin, LIBLINEAR: A library for large linear classification, Journal of Machine Learning Research 9 (2008) 1871–1874.
  * Fricker [2016] R. J. Fricker, False positives are statistically inevitable, Science 351 (2016) 569–570.
  * Gear and Kevrekidis [2005] C. Gear, I. Kevrekidis, Constraint-defined manifolds: a legacy code approach to low-dimensional computation, Journal of Scientific Computing 25 (1) (2005) 17–28.
  * Gibbs [1902] J. Gibbs, Elementary Principles in Statistical Mechanics, developed with especial reference to the rational foundation of thermodynamics, Dover Publications, New York, 1960 [1902].
  * Glorot and Bengio [2010] X. Glorot, Y. Bengio, Understanding the difficulty of training deep feedforward neural networks, in: Proc. of the 13th International Conference on Arificial Intelligence and Statistics (AISTATS), vol. 9, 249–256, 2010.
  * Gorban [2007] A. Gorban, Order-disorder separation: Geometric revision, Physica A 374 (2007) 85–102.
  * Gorban et al. [2016b] A. Gorban, I. Tyukin, D. Prokhorov, K. Sofeikov, Approximation with random bases: Pro et Contra, Information Sciences 364–365 (2016b) 129–145.
  * Gorban et al. [2016a] A. Gorban, I. Tyukin, I. Romanenko, The Blessing of Dimensionality: Separation Theorems in the Thermodynamic Limit, a talk given at TFMST 2016, 2nd IFAC Workshop on Thermodynamic Foundations of Mathematical Systems Theory. September 28-30, 2016, Vigo, Spain, 2016a.
  * Gromov [1999] M. Gromov, Metric Structures for Riemannian and non-Riemannian Spaces. With appendices by M. Katz, P. Pansu, S. Semmes. Translated from the French by Sean Muchael Bates, Birkhauser, Boston, MA, 1999\.
  * Gromov [2003] M. Gromov, Isoperimetry of Waists and Concentration of Maps, GAFA, Geomteric and Functional Analysis 13 (2003) 178–215.
  * Hansen and Salamon [1990] L. K. Hansen, P. Salamon, Neural Network Ensembles, IEEE Trans. on Patt. Analyis and Mach. Intel. 12 (10) (1990) 993–1001.
  * He et al. [2016] K. He, X. Zhang, S. Ren, J. Sun, Deep Residual Learning for Image Recognition, in: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 770–778, arXiv:1512.03385, 2016.
  * Ho [1995] T. K. Ho, Random Decision Forests, in: Proc. of the 3rd International Conference on Document Analysis and Recognition, 993–1001, 1995.
  * Ho [1998] T. K. Ho, The Random Subspace Method for Constructing Decision Forests, IEEE Trans. on Patt. Analyis and Mach. Intel. 20 (8) (1998) 832–844.
  * Ison et al. [2015] M. Ison, R. Quian Quiroga, I. Fried, Rapid encoding of new memories by individual neurons in the human brain, Neuron 87 (1) (2015) 220–230.
  * Jackson [1993] D. Jackson, Stopping Rules in Principal Components Analysis: A Comparison of Heuristical and Statistical Approaches, Ecology 74 (8) (1993) 2204–2214.
  * Jia [2013] Y. Jia, Caffe: An open source convolutional architecture for fast feature embedding, http://caffe.berkeleyvision.org/, 2013\.
  * Johnson and Lindenstrauss [1984] W. B. Johnson, J. Lindenstrauss, Extensions of Lipschitz mappings into a Hilbert space, Contemporary mathematics 26 (1) (1984) 189–206.
  * Krizhevsky et al. [2012] A. Krizhevsky, I. Sutskever, G. Hinton, ImageNet Classification with Deep Convolutional Neural Networks, in: F. Pereira, C. J. C. Burges, L. Bottou, K. Q. Weinberger (Eds.), Advances in Neural Information Processing Systems 25, Curran Associates, Inc., 1097–1105, 2012.
  * Kuznetsova et al. [2015] A. Kuznetsova, S. Hwang, B. Rosenhahn, L. Sigal, Expanding object detector’s horizon: Incremental learning framework for object detection in videos, in: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 28–36, 2015.
  * Lévy [1951] P. Lévy, Problèmes concrets d’analyse fonctionnelle, Gauthier-Villars, Paris, second edn., 1951\.
  * Macarthur [1957] R. Macarthur, On the relative abundance of bird species, Proc. Nat. Acad. of Sci. 43 (3) (1957) 293–295.
  * Milman and Schechtman [2009] V. D. Milman, G. Schechtman, Asymptotic theory of finite dimensional normed spaces: Isoperimetric inequalities in riemannian manifolds, vol. 1200 of _Lecture Notes in Mathematics_ , Springer, 2009.
  * Misra et al. [2015] I. Misra, A. Shrivastava, M. Hebert, Semi-supervised learning for object detectors from video, in: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3594–3602, 2015.
  * Nguyen et al. [2015] A. Nguyen, J. Yosinski, J. Clune, Deep neural networks are easily fooled: High confidence predictions for unrecognizable images, in: Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 427–436, 2015.
  * Prest et al. [2012] A. Prest, C. Leistner, J. Civera, C. Schmid, V. Ferrari, Learning object class detectors from weakly annotated video, in: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3282–3289, 2012.
  * Quian Quiroga [2012] R. Quian Quiroga, Concept cells: the building blocks of declarative memory functions, Nature Reviews Neuroscience 13 (8) (2012) 587–597.
  * Quian Quiroga et al. [2005] R. Quian Quiroga, L. Reddy, G. Kreiman, C. Koch, I. Fried, Invariant visual representation by single neurons in the human brain, Nature 435 (7045) (2005) 1102–1107.
  * Russakovsky et al. [2014] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, L. Fei-Fei, ImageNet Large Scale Visual Recognition Challenge, Int. J. Comput. Vis. (2014) 1–42DOI:10.1007/s11263-015-0816-y.
  * Scardapane and Wang [2017] S. Scardapane, D. Wang, Randomness in neural networks: an overview, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 7 (2) (2017) e1200.
  * Schaefer [1999] H. Schaefer, Topological Vector Spaces, Springer, New York, 1999\.
  * Simonyan and Zisserman [2015] K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, in: International Conference on Learning Representations, arXiv:1409.1556, 2015.
  * Szegedy et al. [2014] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. J. Goodfellow, R. Fergus, Intriguing properties of neural networks, in: Proc. of International Conference on Learining Representations (ICLR), https://arxiv.org/abs/1312.6199, 2014.
  * Team [2016] D. D. Team, Deeplearning4j: Open-source distributed deep learning for the JVM, Apache Software Foundation License 2.0. http://deeplearning4j.org, 2016
  * Vapnik and Chapelle [2000] V. Vapnik, O. Chapelle, Bounds on error expectation for support vector machines, Neural Computation 12 (9) (2000) 2013–2036.
  * Viskontas et al. [2009] I. Viskontas, R. Quian Quiroga, I. Fried, Human medial temporal lobe neurons respond preferentially to personally relevant images, Proc. Nat. Acad. Sci. 106 (50) (2009) 21329–21334.
  * Wang [2016] D. Wang, Editorial: Randomized algorithms for training neural networks, Information Sciences 364–365 (2016) 126–128.
  * Zheng et al. [2016] S. Zheng, Y. Song, T. Leung, I. Goodfellow, Improving the Robustness of Deep Neural Networks via Stability Training, in: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), https://arxiv.org/abs/1604.04326, 2016.