characteristic polynomial $p\left(\lambda\right)$ of $\mathbf{Q}$ vary
continuously with its coefficients.

Thereby, the eigenvalues $\lambda_{N}\leq\mathbf{\ldots}\leq\lambda_{1}$ of
the joint covariance matrix $\mathbf{Q}$ represent joint variabilities between
all of the feature vectors $k_{\mathbf{x}_{i}}$ and $k_{\mathbf{x}_{j}}$ used
to construct $\mathbf{Q}$, so that each element
$y_{i}\left\|k_{\mathbf{x}_{i}}\right\|y_{j}\left\|k_{\mathbf{x}_{j}}\right\|\cos\theta_{k_{\mathbf{x}_{i}}k_{\mathbf{x}_{j}}}$
of $\mathbf{Q}$ where $y_{i}y_{j}=-1$ describes differences between joint
variabilities of feature vectors $k_{\mathbf{x}_{i}}$ and $k_{\mathbf{x}_{j}}$
that belong to different pattern classes, at which point each element
$\left\|k_{\mathbf{x}_{i}}\right\|\left\|k_{\mathbf{x}_{j}}\right\|\cos\theta_{k_{\mathbf{x}_{i}}k_{\mathbf{x}_{j}}}$
of the joint covariance matrix $\mathbf{Q}$ is correlated with the distance
$\left\|k_{\mathbf{x}_{i}}-k_{\mathbf{x}_{j}}\right\|$ between the loci of
certain feature vectors $k_{\mathbf{x}_{i}}$ and $k_{\mathbf{x}_{j}}$.

Now let the geometric locus of the Wolfe-dual novel principal eigenaxis
$\boldsymbol{\psi}$ be subject to a critical minimum eigenenergy constraint

$\lambda_{1}\left\|\boldsymbol{\psi}\right\|_{\min_{c}}^{2}=\boldsymbol{\psi}_{\max}^{T}\mathbf{Q}\boldsymbol{\psi}_{\max}\equiv\left\|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}$

that is symmetrically and equivalently related to the critical minimum
eigenenergy constraint $\left\|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}$ on
the geometric locus of the primal novel principal eigenaxis
$\boldsymbol{\kappa}$, so that the random quadratic form
$\boldsymbol{\psi}_{\max}^{T}\mathbf{Q}\boldsymbol{\psi}_{\max}$ is
symmetrically and equivalently related to the critical minimum eigenenergy
$\left\|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}$ exhibited by the geometric
locus of the primal novel principal eigenaxis $\boldsymbol{\kappa}$, at which
point the random quadratic form
$\boldsymbol{\psi}_{\max}^{T}\mathbf{Q}\boldsymbol{\psi}_{\max}$, plus the
total allowed eigenenergy $\left\|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}$
and the expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}\right)$
exhibited by the geometric locus of the primal novel eigenaxis
$\boldsymbol{\kappa}$ jointly reach their minimum values.

Correspondingly, let the Wolfe-dual eigenenergy functional

$\max\Xi\left(\boldsymbol{\psi}\right)=\mathbf{1}^{T}\boldsymbol{\psi}-\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi/}2\text{,}$

such that $\boldsymbol{\psi}^{T}\mathbf{y}=0$ and $\psi_{i\ast}>0$, be
maximized by the largest eigenvector $\boldsymbol{\psi}_{\max}$ of the joint
covariance matrix $\mathbf{Q}$ of the random quadratic form
$\boldsymbol{\psi}^{T}\boldsymbol{Q\psi}$

$\boldsymbol{Q\psi}_{\max}=\lambda_{1}\boldsymbol{\psi}_{\max}\text{,}$

so that the random quadratic form
$\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ reaches its minimum value,
at which point the total allowed eigenenergy
$\left\|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}$ and the expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}\right)$
exhibited by the geometric locus of the primal novel principal eigenaxis
$\boldsymbol{\kappa}$ are jointly minimized.

Next, let the Wolfe-dual novel principal eigenaxis $\boldsymbol{\psi}$ and the
primal novel principal eigenaxis $\boldsymbol{\kappa}$ be solutions of the
vector algebra locus equation

$\displaystyle\boldsymbol{\psi}$
$\displaystyle=\lambda_{1}^{-1}\boldsymbol{\psi}^{T}\mathbf{Q}$
$\displaystyle=\lambda_{1}\boldsymbol{\psi}_{\max}^{T}\mathbf{Q}\text{,}$

such that the geometric locus of the Wolfe-dual novel principal eigenaxis
$\boldsymbol{\psi}$ is related to the scaled principal eigenvector
$\lambda_{1}^{-1}\boldsymbol{\psi}_{\max}$ of the joint covariance matrix
$\mathbf{Q}$ of the random quadratic form
$\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ acting on the joint
covariance matrix $\mathbf{Q}$, so that the sides $\boldsymbol{\kappa}_{1}$
and $\boldsymbol{\kappa}_{2}$ of the geometric locus of the novel principal
eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ and the
scale factors $\psi_{1i\ast}$ and $\psi_{2i\ast}$ for the components
$\psi_{1i\ast}\frac{k_{\mathbf{x}_{1i\ast}}}{\left\|k_{\mathbf{x}_{1i\ast}}\right\|}$
and
$\psi_{2i\ast}\frac{k_{\mathbf{x}_{2i\ast}}}{\left\|k_{\mathbf{x}_{2i\ast}}\right\|}$
of the principal eigenvector $\boldsymbol{\psi}_{\max}$ of the joint
covariance matrix $\mathbf{Q}$ of the random quadratic form
$\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ are solutions of the system
of vector algebra locus equations

$\displaystyle\sum\nolimits_{i=1}^{l_{1}}\psi_{1i\ast}$
$\displaystyle=\lambda_{1}^{-1}\sum\nolimits_{i=1}^{l_{1}}k_{\mathbf{x}_{1_{i\ast}}}\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)$
$\displaystyle=\lambda_{1}^{-1}\sum\nolimits_{i=1}^{l_{1}}k_{\mathbf{x}_{1_{i\ast}}}\left(\sum\nolimits_{j=1}^{l_{1}}\psi_{1_{j\ast}}k_{\mathbf{x}_{1_{j\ast}}}-\sum\nolimits_{j=1}^{l_{2}}\psi_{2_{j\ast}}k_{\mathbf{x}_{2_{j\ast}}}\right)\text{,}$

and

$\displaystyle\sum\nolimits_{i=1}^{l_{2}}\psi_{2i\ast}$
$\displaystyle=\lambda_{1}^{-1}\sum\nolimits_{i=1}^{l_{2}}k_{\mathbf{x}_{2_{i\ast}}}\left(\boldsymbol{\kappa}_{2}-\boldsymbol{\kappa}_{1}\right)$
$\displaystyle=\lambda_{1}^{-1}\sum\nolimits_{i=1}^{l_{2}}k_{\mathbf{x}_{2_{i\ast}}}\left(\sum\nolimits_{j=1}^{l_{2}}\psi_{2_{j\ast}}k_{\mathbf{x}_{2_{j\ast}}}-\sum\nolimits_{j=1}^{l_{1}}\psi_{1_{j\ast}}k_{\mathbf{x}_{1_{j\ast}}}\right)\text{,}$

at which point each and every one of the components
$\psi_{1i\ast}\frac{k_{\mathbf{x}_{1i\ast}}}{\left\|k_{\mathbf{x}_{1i\ast}}\right\|}$
and
$\psi_{2i\ast}\frac{k_{\mathbf{x}_{2i\ast}}}{\left\|k_{\mathbf{x}_{2i\ast}}\right\|}$
of the principal eigenvector $\boldsymbol{\psi}_{\max}$ of the joint
covariance matrix $\mathbf{Q}$ of the random quadratic form
$\boldsymbol{\psi}^{T}\boldsymbol{Q\psi}$, along with each and every one of
the components $\psi_{1_{i\ast}}k_{\mathbf{x}_{1_{i\ast}}}$and
$\psi_{2_{i\ast}}k_{\mathbf{x}_{2_{i\ast}}}$of the primal novel principal
eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$, are
subject to deep-seated statistical interconnections with each other, so that
likely locations and likelihood values of extreme vectors
$k_{\mathbf{x}_{1_{i\ast}}}$ and $k_{\mathbf{x}_{2_{i\ast}}}$ are
statistically pre-wired within the components
$\psi_{1i\ast}\frac{k_{\mathbf{x}_{1i\ast}}}{\left\|k_{\mathbf{x}_{1i\ast}}\right\|}$
and
$\psi_{2i\ast}\frac{k_{\mathbf{x}_{2i\ast}}}{\left\|k_{\mathbf{x}_{2i\ast}}\right\|}$
of the principal eigenvector $\boldsymbol{\psi}_{\max}$ of the joint
covariance matrix $\mathbf{Q}$ and the inverted joint covariance matrix
$\mathbf{Q}^{-1}$ associated with the pair of random quadratic forms
$\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ and
$\boldsymbol{\psi}^{T}\mathbf{Q}^{-1}\boldsymbol{\psi}$, as well as the
components $\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ and
$\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$ of the geometric locus of the
novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ of the
minimum risk binary classification system $k_{\mathbf{s}}\boldsymbol{\kappa}+$
$\boldsymbol{\kappa}_{0}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$,
such that each and every one of the statistical interconnections is regulated
by the eigenvalues $\lambda_{N}\leq\mathbf{\ldots}\leq\lambda_{1}$ and
$\lambda_{N}^{-1}\leq\mathbf{\ldots}\leq\lambda_{1}^{-1}$ of the joint
covariance matrix $\mathbf{Q}$ and the inverted joint covariance matrix
$\mathbf{Q}^{-1}$ associated with the pair of random quadratic forms
$\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ and
$\boldsymbol{\psi}^{T}\mathbf{Q}^{-1}\boldsymbol{\psi}$.

Furthermore, let the geometric locus of the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ be the
solution of the vector algebra locus equation that represents the geometric
locus of the decision boundary of the minimum risk binary classification
system

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\boldsymbol{\kappa}+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)=0\text{,}$

along with the vector algebra locus equation that represents the geometric
locus of the decision border of the decision region $Z_{1}$ of the system

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)=+1\text{,}$

and the vector algebra locus equation that represents the geometric locus of
the decision border of the decision region $Z_{2}$ of the system

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)=-1\text{,}$

so that each and every one of the points $\mathbf{s}$ that lies on the
geometric loci of the decision boundary and the pair of symmetrically
positioned decision borders of the decision regions $Z_{1}$ and $Z_{2}$ of the
minimum risk binary classification system exclusively reference the geometric
locus of the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$, at
which point the geometric locus of the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$
represents an eigenaxis of symmetry that spans the decision space $Z=Z_{1}\cup
Z_{2}$ of the system.

It follows that the geometric locus of the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\boldsymbol{\
}$is an exclusive principal eigen-coordinate system that contains all of the
covariance and distribution information—for all of the extreme vectors
$k_{\mathbf{x}_{1_{i\ast}}}$ and $k_{\mathbf{x}_{2_{i\ast}}}$—relative to the
covariance and distribution information for a given collection
$\left\\{\mathbf{x}_{i}\right\\}_{i=1}^{N}$ of feature vectors
$\mathbf{x}_{i}$, so that each principal eigenaxis component
$\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ and
$\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$ that lies on the novel
principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}{\large-}\boldsymbol{\kappa}_{2}$
determines a likely location for a correlated extreme point
$\mathbf{x}_{1_{i\ast}}\mathbf{\sim}$ $p\left(\mathbf{x};\omega_{1}\right)$
and $\mathbf{x}_{2_{i\ast}}\mathbf{\sim}$
$p\left(\mathbf{x};\omega_{2}\right)$, and each likelihood component
$\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ and
$\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$ that lies on the novel
principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}{\large-}\boldsymbol{\kappa}_{2}$
determines a likelihood value for the correlated extreme point
$\mathbf{x}_{1_{i\ast}}$ and $\mathbf{x}_{2_{i\ast}}$, where the reproducing
kernel for each extreme point $k_{\mathbf{x}_{1_{i\ast}}}$ and
$k_{\mathbf{x}_{2_{i\ast}}}$ has the preferred form of either
$k_{\mathbf{x}}\left(\mathbf{s}\right)=\left(\mathbf{s}^{T}\mathbf{x}+1\right)^{2}$
or
$k_{\mathbf{x}}\left(\mathbf{s}\right)=\exp\left(-\gamma\left\|\mathbf{s}-\mathbf{x}\right\|^{2}\right)$,
wherein $0.01\leq\gamma\leq 0.1$.

In addition, let the Wolfe-dual novel principal eigenaxis
$\boldsymbol{\psi}=\boldsymbol{\psi}_{1}+\boldsymbol{\psi}_{2}$ and the primal
novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ be
solutions of the vector algebra locus equation

$\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)\boldsymbol{\kappa}=\boldsymbol{\psi}_{1}+\boldsymbol{\psi}_{2}-\left(\sum\nolimits_{i=1}^{l_{1}}\xi_{i}\psi_{1i\ast}+\sum\nolimits_{i=1}^{l_{2}}\xi_{i}\psi_{2_{i_{\ast}}}\right)\text{,}$

so that the geometric locus of the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$
satisfies the geometric locus of the decision boundary of the minimum risk
binary classification system $k_{\mathbf{s}}\boldsymbol{\kappa}+$
$\boldsymbol{\kappa}_{0}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$
in terms of a critical minimum eigenenergy
$\left\|\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right\|_{\min_{c}}^{2}$
and a minimum expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right\|_{\min_{c}}^{2}\right)$,
at which point the total allowed eigenenergy
$\left\|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}$ and the expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}\right)$
exhibited by the minimum risk binary classification system are jointly
regulated by the equilibrium requirement on the dual locus
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ of the
discriminant function at the geometric locus of decision boundary of the
system, so that the geometric locus of the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$
satisfies the law of cosines in the symmetrically balanced manner

$\displaystyle d\left(\mathbf{s}\right)$
$\displaystyle:\left\|\boldsymbol{\kappa}_{1}\right\|_{\min_{c}}^{2}-\left\|\boldsymbol{\kappa}_{1}\right\|\left\|\boldsymbol{\kappa}_{2}\right\|\cos\theta_{\boldsymbol{\kappa}_{1}\boldsymbol{\kappa}_{2}}$
$\displaystyle=\left\|\boldsymbol{\kappa}_{2}\right\|_{\min_{c}}^{2}-\left\|\boldsymbol{\kappa}_{2}\right\|\left\|\boldsymbol{\kappa}_{1}\right\|\cos\theta_{\boldsymbol{\kappa}_{2}\boldsymbol{\kappa}_{1}}$
$\displaystyle=\frac{1}{2}\left\|\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right\|_{\min_{c}}^{2}\text{.}$

Thereby, the minimum risk binary classification system
$k_{\mathbf{s}}\boldsymbol{\kappa}+$
$\boldsymbol{\kappa}_{0}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$
satisfies a state of statistical equilibrium, so that the total allowed
eigenenergy $\left\|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}$ and the
expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}\right)$
exhibited by the system are jointly minimized within the decision space
$Z=Z_{1}\cup Z_{2}$ of the system, at which point the system exhibits the
minimum probability of classification error.

It follows that the components of the dual loci of the novel principal
eigenaxes $\boldsymbol{\psi}=\boldsymbol{\psi}_{1}+\boldsymbol{\psi}_{2}$ and
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ are both
subject to deep-seated statistical interconnections between the elements
$\left\|k_{\mathbf{x}_{i}}\right\|\left\|k_{\mathbf{x}_{j}}\right\|\cos\theta_{k_{\mathbf{x_{i}}}k_{\mathbf{x}_{j}}}$
and the eigenvalues $\lambda_{N}\leq\mathbf{\ldots}\leq\lambda_{1}$ of the
joint covariance matrix $\mathbf{Q}$ of the random quadratic form
$\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$, along with the elements
and the eigenvalues $\lambda_{N}^{-1}\leq\mathbf{\ldots}\leq\lambda_{1}^{-1}$
of the inverted joint covariance matrix $\mathbf{Q}^{-1}$ of the random
quadratic form $\boldsymbol{\psi}^{T}\mathbf{Q}^{-1}\boldsymbol{\psi}$, so
that the geometric locus of the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ is the
principal part of an equivalent representation of the pair of random quadratic
forms $\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ and
$\boldsymbol{\psi}^{T}\mathbf{Q}^{-1}\boldsymbol{\psi}$.

Thereby, the components $\psi_{1_{i\ast}}k_{\mathbf{x}_{1_{i\ast}}}$and
$\psi_{2_{i\ast}}k_{\mathbf{x}_{2_{i\ast}}}$ of the geometric locus of the
novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ are
statistically interconnected with the components
$\psi_{1i\ast}\frac{k_{\mathbf{x}_{1i\ast}}}{\left\|k_{\mathbf{x}_{1i\ast}}\right\|}$
and
$\psi_{2i\ast}\frac{k_{\mathbf{x}_{2i\ast}}}{\left\|k_{\mathbf{x}_{2i\ast}}\right\|}$
of the principal eigenvector $\boldsymbol{\psi}_{\max}$ of the joint
covariance matrix $\mathbf{Q}$ and the inverted joint covariance matrix
$\mathbf{Q}^{-1}$, along with the elements and the eigenvalues of the joint
covariance matrix $\mathbf{Q}$ and the inverted joint covariance matrix
$\mathbf{Q}^{-1}$, so that the geometric locus of the novel principal
eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ is the
exclusive principal eigen-coordinate system of the geometric locus of the
decision boundary

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)=0$

of the minimum risk binary classification system

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0\text{,}$

so that the geometric locus of the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$
satisfies the geometric locus of the decision boundary in terms of a critical
minimum eigenenergy
$\left\|\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right\|_{\min_{c}}^{2}$
and a minimum expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}\right)$
in the following manner

$\displaystyle\left\|\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right\|_{\min_{c}}^{2}$
$\displaystyle=\sum\nolimits_{i=1}^{l_{1}}\psi_{1i\ast}\left(1-\xi_{i}\right)+\sum\nolimits_{i=1}^{l_{2}}\psi_{2i\ast}\left(1-\xi_{i}\right)$
$\displaystyle=\boldsymbol{\psi}_{\max}-\left(\sum\nolimits_{i=1}^{l_{1}}\xi_{i}\psi_{1i\ast}+\sum\nolimits_{i=1}^{l_{2}}\xi_{i}\psi_{2i\ast}\right)\text{,}$

at which point the total allowed eigenenergy
$\left\|\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right\|_{\min_{c}}^{2}$
and the expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right\|_{\min_{c}}^{2}\right)$
exhibited by the geometric locus of the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ are both
regulated by values of the scale factors $\psi_{1i\ast}$ and $\psi_{2i\ast}$
for the components
$\psi_{1i\ast}\frac{k_{\mathbf{x}_{1i\ast}}}{\left\|k_{\mathbf{x}_{1i\ast}}\right\|}$
and
$\psi_{2i\ast}\frac{k_{\mathbf{x}_{2i\ast}}}{\left\|k_{\mathbf{x}_{2i\ast}}\right\|}$
of the principal eigenvector $\boldsymbol{\psi}_{\max}$ of the joint
covariance matrix $\mathbf{Q}$ of the random quadratic form
$\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ and the inverted joint
covariance matrix $\mathbf{Q}^{-1}$ of the random quadratic form
$\boldsymbol{\psi}^{T}\mathbf{Q}^{-1}\boldsymbol{\psi}$, where the
regularization parameters $\xi_{i}=\xi\ll 1$ determine negligible constraints.

It follows that the shape of the decision space $Z=Z_{1}\cup Z_{2}$ of the
minimum risk binary classification system is completely determined by the
geometric locus of the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ of the
system, such that the shape of the geometric locus of the decision boundary
that is represented by the graph of the vector algebra locus equation

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)=0\text{,}$

the shape of the geometric locus of the decision border of the decision region
$Z_{1}$ that is represented by the graph of the vector algebra locus equation

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)=+1\text{,}$

and the shape of the geometric locus of the decision border of the decision
region $Z_{2}$ that is represented by the graph of the vector algebra locus
equation

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)=-1$

are all determined by the geometric locus of the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$, wherein
the exclusive principal eigen-coordinate system
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ is the
principal part of an equivalent representation of the pair of random quadratic
forms $\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ and
$\boldsymbol{\psi}^{T}\mathbf{Q}^{-1}\boldsymbol{\psi}$ in such a manner that
the geometric locus of the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ is the
principal eigenaxis of the geometric loci of the decision boundary and the
pair of symmetrically positioned decision borders of the decision regions
$Z_{1}$ and $Z_{2}$ of the minimum risk binary classification system

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0\text{,}$

so that the geometric locus of the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$
represents an eigenaxis of symmetry that spans the decision space $Z=Z_{1}\cup
Z_{2}$ of the minimum risk binary classification system, such that the
magnitude and the direction of the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ are both
functions of differences between joint variabilities of extreme vectors
$k_{\mathbf{x}_{1_{i\ast}}}$ and $k_{\mathbf{x}_{2_{i\ast}}}$ that belong to
the two classes $\omega_{1}$ and $\omega_{2}$ of random vectors
$\mathbf{x\in}$ $\mathbb{R}^{d}$, at which point the geometric locus of the
novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$
satisfies the geometric locus of the decision boundary in terms of a critical
minimum eigenenergy
$\left\|\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right\|_{\min_{c}}^{2}$
and a minimum expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right\|_{\min_{c}}^{2}\right)$,
so that the uniform properties exhibited by all of the points $\mathbf{s}$
that lie on the geometric locus of the decision boundary are the critical
minimum eigenenergy
$\left\|\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right\|_{\min_{c}}^{2}$
and the minimum risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right\|_{\min_{c}}^{2}\right)$
exhibited by the geometric locus of the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$.

Proof. Theorem 21.1 is proved by conditions expressed by Theorem 7.1 and
Corollary 7.2, along with a constructive proof that demonstrates how a well-
posed constrained optimization algorithm executes the fundamental laws of
binary classification expressed by Theorem 11.1.

As of now, we have proved each and every one of the fundamental laws of binary
classification—that are expressed by Theorem 11.1—by means of a constructive
proof that demonstrates how a well-posed constrained optimization algorithm
executes the fundamental laws. An overview of the constructive proof is
presented below.

##  22 Overview of a Constructive Proof

We have demonstrated how a well-posed constrained optimization algorithm
transforms a collection of labeled feature vectors

$\left(\mathbf{x}_{1}\mathbf{,}y_{1}\right),\ldots,\left(\mathbf{x}_{N}\mathbf{,}y_{N}\right)\in\mathbb{R}^{d}\times
Y,Y=\left\\{\pm 1\right\\}\text{,}$

wherein $N$ feature vectors $\mathbf{x}\in\mathbb{R}^{d}$ are generated by
certain probability density functions $p\left(\mathbf{x};\omega_{1}\right)$
and $p\left(\mathbf{x};\omega_{2}\right)$, into a data-driven mathematical
model of a minimum risk binary classification system

$\displaystyle\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left[\sum\nolimits_{i=1}^{l_{1}}\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}-\sum\nolimits_{i=1}^{l_{2}}\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}\right]$
$\displaystyle+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0\text{,}$

such that the fundamental unknowns are the scale factors $\psi_{1i\ast}$ and
$\psi_{2i\ast}$ for the components
$\psi_{1i\ast}\frac{k_{\mathbf{x}_{1i\ast}}}{\left\|k_{\mathbf{x}_{1i\ast}}\right\|}$
and
$\psi_{2i\ast}\frac{k_{\mathbf{x}_{2i\ast}}}{\left\|k_{\mathbf{x}_{2i\ast}}\right\|}$
of a Wolfe-dual novel principal eigenaxis

$\displaystyle\boldsymbol{\psi}$
$\displaystyle=\sum\nolimits_{i=1}^{l_{1}}\psi_{1i\ast}\frac{k_{\mathbf{x}_{1i\ast}}}{\left\|k_{\mathbf{x}_{1i\ast}}\right\|}+\sum\nolimits_{i=1}^{l_{2}}\psi_{2i\ast}\frac{k_{\mathbf{x}_{2i\ast}}}{\left\|k_{\mathbf{x}_{2i\ast}}\right\|}$
$\displaystyle=\boldsymbol{\psi}_{1}+\boldsymbol{\psi}_{2}$

whose structure and behavior and properties are symmetrically and equivalently
related to the structure and behavior and properties of the primal novel
principal eigenaxis

$\displaystyle\boldsymbol{\kappa}$
$\displaystyle=\sum\nolimits_{i=1}^{l_{1}}\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}-\sum\nolimits_{i=1}^{l_{2}}\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$
$\displaystyle=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$

of the minimum risk binary classification in such a manner that the
discriminant function

$d\left(\mathbf{s}\right)=\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)$

is represented by a geometric locus of a novel principal eigenaxis

$\displaystyle\boldsymbol{\kappa}$
$\displaystyle=\sum\nolimits_{i=1}^{l_{1}}\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}-\sum\nolimits_{i=1}^{l_{2}}\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$
$\displaystyle=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\text{,}$

at which point a dual locus
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ of
likelihood components and principal eigenaxis components
$\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ and
$\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$ represents an exclusive
principal eigen-coordinate system of the geometric locus of the decision
boundary of the system, and also represents an eigenaxis of symmetry that
spans the decision space of the system, wherein each scale factor
$\psi_{1_{i_{\ast}}}$ or $\psi_{2_{i_{\ast}}}$ determines a scaled extreme
vector $\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ or
$\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$ that represents a principal
eigenaxis component—on the exclusive principal eigen-coordinate system
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$—that
determines a likely location for a correlated extreme point
$\mathbf{x}_{1_{i\ast}}\mathbf{\sim}$ $p\left(\mathbf{x};\omega_{1}\right)$ or
$\mathbf{x}_{2_{i\ast}}\mathbf{\sim}$ $p\left(\mathbf{x};\omega_{2}\right)$,
along with a likelihood component that determines a likelihood value for the
correlated extreme point $\mathbf{x}_{1_{i\ast}}$ or $\mathbf{x}_{2_{i\ast}}$,
where the reproducing kernel for each extreme point
$k_{\mathbf{x}_{1_{i\ast}}}$ and $k_{\mathbf{x}_{2_{i\ast}}}$ has the
preferred form of either
$k_{\mathbf{x}}\left(\mathbf{s}\right)=\left(\mathbf{s}^{T}\mathbf{x}+1\right)^{2}$
or
$k_{\mathbf{x}}\left(\mathbf{s}\right)=\exp\left(-\gamma\left\|\mathbf{s}-\mathbf{x}\right\|^{2}\right)$,
wherein $0.01\leq\gamma\leq 0.1$.

By the process of determining the scale factors $\psi_{1i\ast}$ and
$\psi_{2i\ast}$ in accordance with the vector algebra locus equations in
(15.6) and (15.7), we have demonstrated how each scale factor $\psi_{1i\ast}$
and $\psi_{2i\ast}$ maps covariance and distribution information—for a
correlated extreme vector $k_{\mathbf{x}_{1_{i\ast}}}$ and
$k_{\mathbf{x}_{2_{i\ast}}}$—onto the correlated extreme vector
$k_{\mathbf{x}_{1_{i\ast}}}$ and $k_{\mathbf{x}_{2_{i\ast}}}$ in such a manner
that each scale factor $\psi_{1_{i_{\ast}}}$ or $\psi_{2_{i_{\ast}}}$
determines a scaled extreme vector
$\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ or
$\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$ that represents a principal
eigenaxis component on the exclusive principal eigen-coordinate system
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$, along
with a likelihood component, so that each principal eigenaxis component
$\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ or
$\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$ on
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$
determines a likely location for a correlated extreme point
$\mathbf{x}_{1_{i\ast}}$ or $\mathbf{x}_{2_{i\ast}}$, and each likelihood
component $\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ or
$\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$ on
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$
determines a likelihood value for the correlated extreme point
$\mathbf{x}_{1_{i\ast}}$ or $\mathbf{x}_{2_{i\ast}}$.

By the process of determining the vector algebra locus equations of (16.4) -
(16.6), we have demonstrated how the geometric locus of the novel principal
eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ of a
minimum risk binary classification system $k_{\mathbf{s}}\boldsymbol{\kappa}+$
$\boldsymbol{\kappa}_{0}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$
represents an exclusive principal eigen-coordinate system of the geometric
locus of the decision boundary of the system and also represents an eigenaxis
of symmetry for the decision space $Z=Z_{1}\cup Z_{2}$ of the system.

By the conditions expressed in Theorem 21.1, we have demonstrated how a
geometric locus of a novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ is the
principal part of an equivalent representation of a pair of random quadratic
forms $\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ and
$\boldsymbol{\psi}^{T}\mathbf{Q}^{-1}\boldsymbol{\psi}$ associated with a
joint covariance matrix $\mathbf{Q}$ and the inverted joint covariance matrix
$\mathbf{Q}^{-1}$, so that the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ is the
principal eigenaxis of the decision boundary of a minimum risk binary
classification system $k_{\mathbf{s}}\boldsymbol{\kappa}+$
$\boldsymbol{\kappa}_{0}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$,
at which point the geometric locus of the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$
satisfies the geometric locus of the decision boundary in terms of the
critical minimum eigenenergy and the minimum expected risk exhibited by the
minimum risk binary classification system.

Theorem 21.1 also identifies critical interconnections between the intrinsic
components of a minimum risk binary classification system that determine the
statistical structure and the functionality of the discriminant function of
the system, so that the discriminant function of any given minimum risk binary
classification system generalizes in a significant manner and thereby
extrapolates.

By the process of determining the integral equation in (20.10), we have
demonstrated how a data-driven version of the general form of the integral
equation in (11.7) is determined by the constrained optimization algorithm
that resolves the inverse problem of the binary classification of random
vectors. Accordingly, we have demonstrated that any given minimum risk binary
classification system satisfies the law of total allowed eigenenergy for
minimum risk binary classification systems expressed by (11.7) in Theorem
11.1.

Thereby, we have also demonstrated that the geometric locus of the novel
principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ of any
given minimum risk binary classification system
$k_{\mathbf{s}}\boldsymbol{\kappa}+$
$\boldsymbol{\kappa}_{0}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$
satisfies the law of cosines in a symmetrically balanced manner, so that the
minimum risk binary classification system satisfies the law of symmetry for
minimum risk binary classification systems expressed by (11.9) in Theorem
11.1.

By the process of determining the integral equation in (20.12), we have
demonstrated how a data-driven version of the general form of the integral
equation in (11.8) is determined by the constrained optimization algorithm
that resolves the inverse problem of binary classification. Thereby, we have
demonstrated that any given minimum risk binary classification system
satisfies the law of statistical equilibrium for minimum risk binary
classification systems expressed by (11.8) in Theorem 11.1.

In conclusion, we have demonstrated how data-driven versions of the general
forms of the vector algebra locus formulae in (11.1) - (11.9)—which are
expressed in the direct problem of the binary classification of random vectors
by Theorem 11.1—are determined by the constrained optimization algorithm that
resolves the inverse problem of the binary classification of random vectors.

Thereby, we are now in a position to express the inverse problem of the binary
classification of random vectors.

##  23 The Inverse Problem

We have proved Theorem 23.1 by means of a constructive proof that demonstrates
how a well-posed constrained optimization algorithm executes the fundamental
laws of binary classification expressed by Theorem 11.1.

###### Theorem 23.1

Let

(23.1)
$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$

be any given minimum risk binary classification system that is subject to
random inputs $\mathbf{x\in}$ $\mathbb{R}^{d}$ such that $\mathbf{x\sim}$
$p\left(\mathbf{x};\omega_{1}\right)$ and $\mathbf{x\sim}$
$p\left(\mathbf{x};\omega_{2}\right)$, where either $\xi_{i}=\xi=0$ or
$\xi_{i}=\xi\ll 1$, $y_{i}=\pm 1$, and $p\left(\mathbf{x};\omega_{1}\right)$
and $p\left(\mathbf{x};\omega_{2}\right)$ are certain probability density
functions for two classes $\omega_{1}$ and $\omega_{2}$ of random vectors
$\mathbf{x\in}$ $\mathbb{R}^{d}$, where$\ \omega_{1}$ or $\omega_{2}$ is the
true category, satisfying the following geometrical and statistical criteria:

$1$. The discriminant function

(23.2)
$d\left(\mathbf{s}\right)=\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)$

is represented by a geometric locus of a novel principal eigenaxis

(23.3) $\displaystyle\boldsymbol{\kappa}$
$\displaystyle=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$
$\displaystyle=\sum\nolimits_{i=1}^{l_{1}}\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}-\sum\nolimits_{i=1}^{l_{2}}\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$

structured as a locus of signed and scaled extreme vectors
$\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ and
$-\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$, so that a dual locus of
likelihood components and principal eigenaxis components
$\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ and
$\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$ represents an exclusive
principal eigen-coordinate system of the geometric locus of the decision
boundary of the system, and also represents an eigenaxis of symmetry that
spans the decision space $Z=Z_{1}\cup Z_{2}$ of the system, such that each
scale factor $\psi_{1_{i_{\ast}}}$ or $\psi_{2_{i_{\ast}}}$ determines a
scaled extreme vector $\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ or
$\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$ that represents a principal
eigenaxis component that determines a likely location for a correlated extreme
point $\mathbf{x}_{1_{i\ast}}\mathbf{\sim}$
$p\left(\mathbf{x};\omega_{1}\right)$ or $\mathbf{x}_{2_{i\ast}}\mathbf{\sim}$
$p\left(\mathbf{x};\omega_{2}\right)$, along with a likelihood component that
determines a likelihood value for the correlated extreme point
$\mathbf{x}_{1_{i\ast}}$ or $\mathbf{x}_{2_{i\ast}}$, where the reproducing
kernel for each extreme point $k_{\mathbf{x}_{1_{i\ast}}}$ and
$k_{\mathbf{x}_{2_{i\ast}}}$ has the preferred form of either
$k_{\mathbf{x}}\left(\mathbf{s}\right)=\left(\mathbf{s}^{T}\mathbf{x}+1\right)^{2}$
or
$k_{\mathbf{x}}\left(\mathbf{s}\right)=\exp\left(-\gamma\left\|\mathbf{s}-\mathbf{x}\right\|^{2}\right)$,
wherein $0.01\leq\gamma\leq 0.1$;

$2$. The geometric locus of the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ is the
solution of the vector algebra locus equation

(23.4)
$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)=0$

that represents the geometric locus of the decision boundary of the system,
where the expression
$\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}$ represents a locus
of average risk in the decision space $Z=Z_{1}\cup Z_{2}$ of the system, and
the statistic
$\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right):y_{i}=\pm 1$
represents an expected likelihood of observing $l$ extreme vectors
$\left\\{k_{\mathbf{x}_{i\ast}}\right\\}_{i=1}^{l}$ within the decision space
$Z=Z_{1}\cup Z_{2}$, so that all of the points $\mathbf{s}$ that lie on the
geometric locus of the decision boundary exclusively reference the novel
principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$, as well
as the vector algebra locus equations

(23.5)
$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)=+1$

and

(23.6)
$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)=-1$

that represent the geometric loci of the decision borders of the corresponding
decision regions $Z_{1}$ and $Z_{2}$ of the system, so that all of the points
$\mathbf{s}$ that lie on the geometric loci of the decision borders
exclusively reference the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$.

Thereby, the geometric locus of the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$
represents an eigenaxis of symmetry that spans the decision space $Z=Z_{1}\cup
Z_{2}$ of the minimum risk binary classification system

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0\text{,}$

at which point the shape of the decision space $Z=Z_{1}\cup Z_{2}$ is
completely determined by the exclusive principal eigen-coordinate system
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$;

$3$. The discriminant function is the solution of the integral equation

(23.7) $\displaystyle f_{1}\left(d\left(\mathbf{s}\right)\right)$
$\displaystyle:\int_{Z_{1}}\boldsymbol{\kappa}_{1}d\boldsymbol{\kappa}_{1}+\int_{Z_{2}}\boldsymbol{\kappa}_{1}d\boldsymbol{\kappa}_{1}+\delta\left(y\right)\lambda_{1}^{-1}\sum\nolimits_{i=1}^{l_{1}}k_{\mathbf{x}_{1_{i\ast}}}\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)$
$\displaystyle=\int_{Z_{1}}\boldsymbol{\kappa}_{2}d\boldsymbol{\kappa}_{2}+\int_{Z_{2}}\boldsymbol{\kappa}_{2}d\boldsymbol{\kappa}_{2}+\delta\left(y\right)\lambda_{1}^{-1}\sum\nolimits_{i=1}^{l_{2}}k_{\mathbf{x}_{2_{i\ast}}}\left(\boldsymbol{\kappa}_{2}-\boldsymbol{\kappa}_{1}\right)\text{,}$

over the decision space $Z=Z_{1}\cup Z_{2}$ of the minimum risk binary
classification system

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0\text{,}$

where
$\delta\left(y\right)=\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)$,
so that the total allowed eigenenergy
$\left\|\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right\|_{\min_{c}}^{2}$
and the expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right\|_{\min_{c}}^{2}\right)$
exhibited by the system are jointly regulated by the equilibrium requirement
on the dual locus
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ of the
discriminant function at the geometric locus of the decision boundary of the
system

$\displaystyle d\left(\mathbf{s}\right)$
$\displaystyle:\left\|\boldsymbol{\kappa}_{1}\right\|_{\min_{c}}^{2}-\left\|\boldsymbol{\kappa}_{1}\right\|\left\|\boldsymbol{\kappa}_{2}\right\|\cos\theta_{\boldsymbol{\kappa}_{1}\boldsymbol{\kappa}_{2}}$
$\displaystyle=\left\|\boldsymbol{\kappa}_{2}\right\|_{\min_{c}}^{2}-\left\|\boldsymbol{\kappa}_{2}\right\|\left\|\boldsymbol{\kappa}_{1}\right\|\cos\theta_{\boldsymbol{\kappa}_{2}\boldsymbol{\kappa}_{1}}$
$\displaystyle\equiv\frac{1}{2}\left\|\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right\|_{\min_{c}}^{2}\text{,}$

at which point the dual locus
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ of the
discriminant function satisfies the geometric locus of the decision boundary
in terms of a critical minimum eigenenergy
$\left\|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}$ and a minimum expected
risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}\right)$
in such a manner that regions of counter risks of the system are symmetrically
balanced with regions of risks of the system, so that critical minimum
eigenenergies
$\left\|\psi_{1_{i\ast}}k_{\mathbf{x}_{1_{i\ast}}}\right\|_{\min_{c}}^{2}$
exhibited by principal eigenaxis components
$\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ on side
$\boldsymbol{\kappa}_{1}$ of the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$—that
determine probabilities of finding extreme points $\mathbf{x}_{1_{i\ast}}$
located throughout the decision space $Z=Z_{1}\cup Z_{2}$ of the system, are
symmetrically balanced with critical minimum eigenenergies
$\left\|\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}\right\|_{\min_{c}}^{2}$
exhibited by principal eigenaxis components
$\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$ on side
$\boldsymbol{\kappa}_{2}$ of the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$—that
determine probabilities of finding extreme points $\mathbf{x}_{2_{i\ast}}$
located throughout the decision space $Z=Z_{1}\cup Z_{2}$ of the system;

$4$. The discriminant function minimizes the integral equation

(23.8) $\displaystyle f_{2}\left(d\left(\mathbf{s}\right)\right)$
$\displaystyle:\int_{Z_{1}}\boldsymbol{\kappa}_{1}d\boldsymbol{\kappa}_{1}-\int_{Z_{1}}\boldsymbol{\kappa}_{2}d\boldsymbol{\kappa}_{2}+\delta\left(y\right)\lambda_{1}^{-1}\sum\nolimits_{i=1}^{l_{1}}k_{\mathbf{x}_{1_{i\ast}}}\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)$
$\displaystyle=\int_{Z_{2}}\boldsymbol{\kappa}_{2}d\boldsymbol{\kappa}_{2}-\int_{Z_{2}}\boldsymbol{\kappa}_{1}d\boldsymbol{\kappa}_{1}+\delta\left(y\right)\lambda_{1}^{-1}\sum\nolimits_{i=1}^{l_{2}}k_{\mathbf{x}_{2_{i\ast}}}\left(\boldsymbol{\kappa}_{2}-\boldsymbol{\kappa}_{1}\right)\text{,}$

over the decision regions $Z_{1}$ and $Z_{2}$ of the minimum risk binary
classification system
$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$,
where
$\delta\left(y\right)=\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)$,
so that the system satisfies a state of statistical equilibrium such that the
total allowed eigenenergy
$\left\|\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right\|_{\min_{c}}^{2}$
and the expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right\|_{\min_{c}}^{2}\right)$
exhibited by the system are jointly minimized within the decision space
$Z=Z_{1}\cup Z_{2}$ of the system, at which point critical minimum
eigenenergies
$\left\|\psi_{1_{i\ast}}k_{\mathbf{x}_{1_{i\ast}}}\right\|_{\min_{c}}^{2}$ and
$\left\|\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}\right\|_{\min_{c}}^{2}$
exhibited by corresponding principal eigenaxis components
$\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ and
$\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$ on side
$\boldsymbol{\kappa}_{1}$ and side $\boldsymbol{\kappa}_{2}$ of the novel
principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ are
minimized throughout the decision regions $Z_{1}$ and $Z_{2}$ of the system,
so that regions of counter risks and risks of the system—located throughout
the decision region $Z_{1}$ of the system—are symmetrically balanced with
regions of counter risks and risks of the system—located throughout the
decision region $Z_{2}$ of the system.

Thereby, the minimum risk binary classification system

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$

satisfies a state of statistical equilibrium so that the total allowed
eigenenergy and the expected risk exhibited by the system are jointly
minimized within the decision space $Z=Z_{1}\cup Z_{2}$ of the system, at
which point the system exhibits the minimum probability of classification
error for any given random vectors $\mathbf{x\in}$ $\mathbb{R}^{d}$ such that
$\mathbf{x\sim}$ $p\left(\mathbf{x};\omega_{1}\right)$ and $\mathbf{x\sim}$
$p\left(\mathbf{x};\omega_{2}\right)$;

$5$. The geometric locus of the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$
satisfies the law of cosines in the symmetrically balanced manner

(23.9)
$\displaystyle\frac{1}{2}\left\|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}$
$\displaystyle=\left\|\boldsymbol{\kappa}_{1}\right\|_{\min_{c}}^{2}-\left\|\boldsymbol{\kappa}_{1}\right\|\left\|\boldsymbol{\kappa}_{2}\right\|\cos\theta_{\boldsymbol{\kappa}_{1}\boldsymbol{\kappa}_{2}}$
$\displaystyle=\left\|\boldsymbol{\kappa}_{2}\right\|_{\min_{c}}^{2}-\left\|\boldsymbol{\kappa}_{2}\right\|\left\|\boldsymbol{\kappa}_{1}\right\|\cos\theta_{\boldsymbol{\kappa}_{2}\boldsymbol{\kappa}_{1}}\text{,}$

where $\theta$ is the angle between $\boldsymbol{\kappa}_{1}$ and
$\boldsymbol{\kappa}_{2}$, so that the geometric locus of the novel principal
eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$
represents an eigenaxis of symmetry that exhibits symmetrical dimensions and
densities, such that the magnitude and the direction of the novel principal
eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ are both
functions of differences between joint variabilities of extreme vectors
$k_{\mathbf{x}_{1_{i\ast}}}$ and $k_{\mathbf{x}_{2_{i\ast}}}$ that belong to
the two classes $\omega_{1}$ and $\omega_{2}$ of random vectors
$\mathbf{x\in}$ $\mathbb{R}^{d}$, at which point the critical minimum
eigenenergy $\left\|\boldsymbol{\kappa}_{1}\right\|_{\min_{c}}^{2}$ exhibited
by side $\boldsymbol{\kappa}_{1}$ is symmetrically balanced with the critical
minimum eigenenergy $\left\|\boldsymbol{\kappa}_{2}\right\|_{\min_{c}}^{2}$
exhibited by side $\boldsymbol{\kappa}_{2}$

$\left\|\boldsymbol{\kappa}_{1}\right\|_{\min_{c}}^{2}=\left\|\boldsymbol{\kappa}_{2}\right\|_{\min_{c}}^{2}\text{,}$

the length of side $\boldsymbol{\kappa}_{1}$ equals the length of side
$\boldsymbol{\kappa}_{2}$

$\left\|\boldsymbol{\kappa}_{1}\right\|=\left\|\boldsymbol{\kappa}_{2}\right\|\text{,}$

and counteracting and opposing forces and influences of the minimum risk
binary classification system

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$

are symmetrically balanced with each other about the geometric center of the
locus of the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$

$\displaystyle\left\|\boldsymbol{\kappa}_{1}\right\|\left(\sum\nolimits_{i=1}^{l_{1}}\operatorname{comp}_{\overrightarrow{\boldsymbol{\kappa}_{1}}}\left(\overrightarrow{\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}}\right)-\sum\nolimits_{i=1}^{l_{2}}\operatorname{comp}_{\overrightarrow{\boldsymbol{\kappa}_{1}}}\left(\overrightarrow{\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}}\right)\right)$
$\displaystyle=\left\|\boldsymbol{\kappa}_{2}\right\|\left(\sum\nolimits_{i=1}^{l_{2}}\operatorname{comp}_{\overrightarrow{\boldsymbol{\kappa}_{2}}}\left(\overrightarrow{\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}}\right)-\sum\nolimits_{i=1}^{l_{1}}\operatorname{comp}_{\overrightarrow{\boldsymbol{\kappa}_{2}}}\left(\overrightarrow{\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}}\right)\right)\text{,}$

whereon the statistical fulcrum of the novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ is
located.

Thereby, counteracting and opposing components of critical minimum
eigenenergies related to likely locations of extreme points from class
$\omega_{1}$ and class $\omega_{2}$ that determine regions of counter risks
and risks of the system—along the dual locus of side
$\boldsymbol{\kappa}_{1}$—are symmetrically balanced with counteracting and
opposing components of critical minimum eigenenergies related to likely
locations of extreme points from class $\omega_{2}$ and class $\omega_{1}$
that determine regions of counter risks and risks of the system—along the dual
locus of side $\boldsymbol{\kappa}_{2}$;

$6$. The center of total allowed eigenenergy and expected risk of the minimum
risk binary classification system

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\left(1-\xi_{i}\right)\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$

is located at the geometric center of the locus of the novel principal
eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ of the
system, whereon the statistical fulcrum of the system is located;

Then the minimum risk binary classification system acts to jointly minimize
its eigenenergy and risk by locating a point of equilibrium, at which point
critical minimum eigenenergies exhibited by the system are symmetrically
concentrated in such a manner that the geometric locus of the novel principal
eigenaxis of the system is an eigenaxis of symmetry that exhibits symmetrical
dimensions and densities, so that the dual locus of the discriminant function
of the system is in statistical equilibrium at the geometric locus of the
decision boundary of the system, such that counteracting and opposing forces
and influences of the system are symmetrically balanced with each other—about
the geometric center of the locus of the novel principal eigenaxis—whereon the
statistical fulcrum of the system is located.

Thereby, the minimum risk binary classification system satisfies a state of
statistical equilibrium so that the total allowed eigenenergy and the expected
risk exhibited by the system are jointly minimized within the decision space
of the system, at which point the system exhibits the minimum probability of
classification error.

The general locus formula that resolves the inverse problem of the binary
classification of random vectors—that is expressed by Theorem 23.1—is readily
generalized to applied minimum risk multiclass classification systems.

###  23.1 Applied Minimum Risk Classification Systems

Corollary 23.2 generalizes the fundamental laws of binary classification
expressed by Theorem 23.1 to applied minimum risk multiclass classification
systems.

###### Corollary 23.2

Any given minimum risk multiclass classification system that is subject to $M$
sources of random vectors $\mathbf{x\in}$ $\mathbb{R}^{d}$ is determined by
$M$ ensembles of $M-1$ minimum risk binary classification systems, such that
each ensemble is determined by an architecture wherein one class is compared
with all of the other $M-1$ classes, so that every one of the $M-1$ minimum
risk binary classification systems in each and every one of the $M$ ensembles
satisfies the geometrical and statistical criteria expressed by Theorem 23.1.

Thereby, the minimum risk multiclass classification system satisfies a state
of statistical equilibrium so that the total allowed eigenenergy and the
expected risk exhibited by the system are jointly minimized within the
decision space of the system, at which point the system exhibits the minimum
probability of classification error.

Proof. Corollary 23.2 is proved by Theorem 23.1 and the superposition
principle [39]—since any given applied minimum risk multiclass classification
system is based on a “one versus all” architecture.

We have finally reached a position where we have completed our treatise on the
fundamental problem of the binary classification of random vectors. Our major
findings are presented below.

##  24 Major Findings

We have covered a lot of ground in this treatise on the fundamental problem of
the binary classification of random vectors. We now summarize our major
findings.

We have proved that Bayes’ decision rule—which is considered the gold standard
for binary and multiclass classification tasks—does not satisfy the conditions
of Bayes’ theorem, wherein identical random vectors generated by distinct
probability density functions account for the same effect exhibited by a
binary classification system. Thereby, we have demonstrated that Bayes’
decision rule constitutes an ill-posed rule for the direct problem of the
binary classification of random vectors—at which point the direct problem was
recognized to be an ill-posed problem.

We have developed a well-posed rule for the direct problem of the binary
classification of random vectors. Namely, we have derived a general locus
formula that resolves the direct problem of the binary classification of
random vectors by enlarging the complexity of a likelihood ratio test—that is
based on the _maximum likelihood criterion_ —which constitutes a well-posed
variant of “Bayes’ decision rule” for binary classification systems.

Thereby, we have developed a general locus formula for finding discriminant
functions of minimum risk binary classification systems that has the general
form of a system of fundamental locus equations of binary classification,
subject to distinctive geometrical and statistical conditions for a minimum
risk binary classification system in statistical equilibrium, so that certain
random vectors have coordinates that are solutions of the locus equations.

Thus, we have devised a theoretical model of a minimum risk binary
classification system that is based on the mathematical structure of the
operator of the system.

As a result, we have _uncovered the black box_ of a theoretical model of a
_minimum risk binary classification system_.

We have also uncovered a machine learning algorithm that resolves the inverse
problem of the binary classification of random vectors by identifying novel
and extremely unobvious processes—which include a novel principal eigen-
coordinate transform algorithm—that are executed by a well-posed variant of
the constrained optimization algorithm that is used by support vector machines
to learn nonlinear decision boundaries.

It was seen that the machine learning algorithm _finds a system_ of
fundamental _locus equations_ of binary classification, subject to distinctive
geometrical and statistical conditions for a minimum risk binary
classification system in statistical equilibrium—that is satisfied by certain
random vectors—at which point data-driven versions of the general forms of the
fundamental locus equations were seen to be determined by distinctive
algebraic and geometric interconnections between all of the random vectors and
the components of the minimum risk binary classification system.

It was also seen that the data-driven version of the general locus formula
executes precise mathematical conditions that _statistically_ _pre-wire_
important _generalizations_ within the _discriminant function_ of a minimum
risk binary classification system—so that the discriminant function
generalizes and thereby _extrapolates_ in a significant manner.

Thereby, it was seen that the structure and function of minimum risk binary
classification systems are intimately intertwined, such that the _structure_
of a discriminant function of a minimum risk binary classification system is
essential for its _functionality_ —which includes the ability of the
discriminant function to _generalize_.

Moreover, it was seen that the machine learning algorithm finds discriminant
functions—of minimum risk binary classification systems—by executing a novel
principal eigen-coordinate transform algorithm.

Thus, we have revealed a machine learning algorithm that determines the
mathematical structure of an operator of a minimum risk binary classification
system, such that the mathematical structure of the operator of the learning
machine _is aligned with_ the mathematical structure of the operator of the
theoretical model.

As a result, we have _uncovered the black box_ of an applied model of a
_minimum risk binary classification system_.

We have demonstrated that reproducing kernels are fundamental components of
minimum risk binary classification systems, such that certain types of
reproducing kernels replace random vectors with second-order curves—formed by
first and second degree vector components—that are more or less sinuous and
thereby preserve topological properties of vectors in Hilbert space. We have
also demonstrated that geometric loci of linear and quadratic decision
boundaries are both well-approximated by such second order curves in certain
reproducing kernel Hilbert spaces.

We have proved, from first principles, that any given minimum risk binary
classification system that is subject to random vectors has a certain
statistical structure and exhibits certain statistical behavior and
properties.

Thereby, we have revealed a machine learning algorithm that determines minimum
risk binary classification systems whose statistical structure and behavior
and properties _match_ the statistical structure and behavior and properties
exhibited by the theoretical model of a minimum risk binary classification
system.

Correspondingly, we have devised a _mathematical system_ whose statistical
structure and behavior and properties _models_ fundamental aspects of a
minimum risk binary classification system—which is subject to random vectors.
The model represents a discriminant function, a decision boundary, an
exclusive principal eigen-coordinate system and an eigenaxis of symmetry—that
spans the decision space—of a minimum risk binary classification system, so
that the exclusive principal eigen-coordinate system connects the discriminant
function to the decision boundary of the system, at which point the
discriminant function, the exclusive principal eigen-coordinate system and the
eigenaxis of symmetry are each represented by a geometric locus of a novel
principal eigenaxis—which has the structure of a dual locus of likelihood
components and principal eigenaxis components.

We used the model of a minimum risk binary classification system that is
outlined above to explain how a discriminant function extrapolates—and thereby
generalizes in a significant manner. We also used the model to explain how a
minimum risk binary classification system acts to minimize its risk. Even
more, we used the model to predict error rates exhibited by minimum risk
binary classification systems.

Thereby, we have proved that discriminant functions of minimum risk binary
classification systems extrapolate, and thereby generalize in a very
nontrivial manner—because the important generalizations for a minimum risk
binary classification system are statistically pre-wired within the geometric
locus of the novel principal eigenaxis of the system—by means a system of
fundamental locus equations of binary classification, subject to distinctive
geometrical and statistical conditions for a minimum risk binary
classification system in statistical equilibrium—that is satisfied by certain
random vectors.

Equally important, we used the model to _predict behavior_ that we have not
been aware of. We used the model to predict that any given minimum risk binary
classification system acts to jointly minimize its eigenenergy and risk by
locating a point of equilibrium—at which point critical minimum eigenenergies
exhibited by the system are symmetrically concentrated—so that the
discriminant function of the system is in statistical equilibrium at the
decision boundary of the system, such counteracting and opposing forces and
influences of the system are symmetrically balanced with each other—about the
geometric center of the locus of the novel principal eigenaxis of the
system—whereon the statistical fulcrum of the system is located.

Thereby, we have proved that any given minimum risk binary classification
system satisfies a state of statistical equilibrium so that the total allowed
eigenenergy and the expected risk exhibited by the system are jointly
minimized within the decision space of the system, at which point the system
exhibits the minimum probability of classification error.

We have demonstrated that the use of scalar-valued cost functions grossly
oversimplifies the complexity of the fundamental problem of finding
discriminant functions of minimum risk binary classification systems. We have
shown that finding a discriminant function of a minimum risk binary
classification system involves optimizing a vector-value cost function—in
accordance with a well-posed eigenenergy functional—so that the total allowed
eigenenergy exhibited by the minimum risk binary classification system is
regulated by critical minimum eigenenergy constraints on a pair of primal and
dual novel principal eigenaxes that are symmetrically and equivalently related
to each other. Thereby, we have demonstrated that the use of a vector-valued
cost function is essential for finding discriminant functions of minimum risk
binary classification systems—that are subject to random vectors.

We have demonstrated that the overall structure and behavior and properties
exhibited by any given minimum risk binary classification system are
determined by elegant, deep-seated interconnections—between intrinsic
components of the system and the random vectors used to infer the values of
the parameters of the system—such that the parameters of the system
characterize the overall structure and behavior and properties of the system.

Correspondingly we have demonstrated that the essential information content of
any given training data set—that is used to find a discriminant function of a
minimum risk binary classification system—is contained within the
eigenstructures of the data set, such that all of the individual feature
vectors ‘add up’ to a complete and sufficient eigenstructure, so that all of
the individual feature vectors ‘speak for themselves’—at which point joint
variabilities between all of the feature vectors are ‘accounted for.’

We have devised a mathematical framework for the direct problem and the
inverse problem of the binary classification of random vectors by devising
novel geometric locus methods in Hilbert __ spaces—within statistical
frameworks—that fruitfully treat fundamental locus problems in binary
classification, where the Hilbert spaces are reproducing kernel Hilbert spaces
that have certain reproducing kernels. Thereby, we have demonstrated that
finding discriminant functions of minimum risk binary classification systems
is essentially a deep-seated locus problem in binary classification—situated
far beneath the surface—at which point underlying aspects of the problem are
subtle and extremely unobvious conditions.

We have demonstrated that the well-posed constrained optimization algorithm
that resolves the inverse problem of the binary classification of random
vectors executes novel and elegant processes—which include a novel principal
eigen-coordinate transform algorithm—that represent the solution for finding
discriminant functions of minimum risk binary classification systems, at which
point the direct problem is transformed into a feasible one.

Finally, we have demonstrated that the inverse problem of the binary
classification of random vectors is directly related to the forward problem of
the binary classification of random vectors—by fundamental laws of binary
classification that discriminant functions of minimum risk binary
classification systems are subject to.

The major findings outlined above promote new insights into fundamental
problems in both machine learning and data-driven mathematical modeling
applications.

##  25 New Insights

We have discovered that the general problem of the binary classification of
random vectors is essentially a deep-seated locus problem in binary
classification that is situated far beneath the surface—at which point
underlying aspects of the problem are subtle and extremely unobvious
conditions.

Correspondingly, we have discovered that the general problem of the binary
classification of random vectors is a statistical coordinate transform
problem, so that the general problem is resolved by a suitable change of the
basis of an intrinsic coordinate system of a locus equation of the decision
boundary of a minimum risk binary classification system, so that an exclusive
principal eigen-coordinate system—that provides dual representation of the
discriminant function, the intrinsic coordinate system of the decision
boundary, and an eigenaxis of symmetry that spans the decision space of the
minimum risk binary classification system—is generated by a novel principal
eigen-coordinate transform algorithm.

Even more, we have discovered that the general problem of the binary
classification of random vectors is also a system identification problem, so
that the overall statistical structure and behavior and properties of a binary
classification system are determined by transforming a collection of
observations into a data-driven mathematical model that represents fundamental
aspects of the system.

We have also discovered that solving the system identification problem for the
general problem of the binary classification of random vectors involves
solving a direct (forward) problem—which entails determining a fully specified
mathematical model of a binary classification system whose solution is used to
predict some type of system behavior.

As a result, we have obtained new insights into long-standing and deep-seated
problems in both machine learning and data-driven mathematical modeling
applications.

We have discovered that the direct problem of the binary classification of
random vectors is an ill-posed problem, wherein identical random vectors
generated by distinct probability density functions account for the same
effect exhibited by a binary classification system.

We have discovered that resolving the general problem of the binary
classification of random vectors requires solving a system identification
problem, so that a deep-seated statistical dilemma is resolved by a well-posed
constrained optimization algorithm that executes a novel principal eigen-
coordinate transform algorithm.

We have discovered that resolving the bias and variance dilemma—for the
fundamental problem of finding discriminant functions of minimum risk binary
classification systems, subject to random vectors—requires solving a data-
driven mathematical modeling problem that is fruitfully treated by novel
geometric locus methods in Hilbert __ spaces—within statistical frameworks.

Correspondingly, we have discovered that resolving the bias and variance
dilemma—for the fundamental problem of finding discriminant functions of
minimum risk binary classification systems, subject to random vectors—requires
finding a natural solution that determines an exclusive principal eigen-
coordinate system of the geometric locus of the decision boundary of a minimum
risk binary classification system.

Thereby, we have discovered that the general problem of the binary
classification of random vectors is a novel principal eigen-coordinate
transformation problem, so that the general problem is resolved by a suitable
change of coordinate system.

We have discovered a data-driven version of a general locus formula for
finding discriminant functions of minimum risk binary classification systems
that has the general form of a system of fundamental locus equations of binary
classification, subject to distinctive geometrical and statistical conditions
for a minimum risk binary classification system in statistical equilibrium, so
that certain random vectors have coordinates that are solutions of the locus
equations. Thereby, we have discovered that the data-driven version of the
general locus formula executes precise mathematical conditions that
_statistically_ _pre-wire_ important _generalizations_ within the
_discriminant function_ of a minimum risk binary classification system—so that
the discriminant function generalizes and thereby _extrapolates_ in a
significant manner.

Thus, we have discovered that the _structure_ of a discriminant function of a
minimum risk binary classification system is essential for its _functionality_
—which includes the ability of the discriminant function to _generalize_.

We have discovered that the “cost” of finding a discriminant function of a
minimum risk binary classification system—subject to random vectors—is the
critical minimum eigenenergy that is necessary for the system to achieve a
state of statistical equilibrium, at which point critical minimum
eigenenergies exhibited by the system are symmetrically concentrated in such a
manner that the geometric locus of the novel principal eigenaxis of the system
represents an eigenaxis of symmetry that exhibits symmetrical dimensions and
densities, so that counteracting and opposing forces and influences of the
system are symmetrically balanced with each other—about the geometric center
of the locus of the novel principal eigenaxis—whereon the statistical fulcrum
of the system is located.

Correspondingly, we have discovered that the total allowed eigenenergy that is
exhibited by a minimum risk binary classification system accounts for right
and wrong decisions made by the system. Thereby, we have discovered that
vector-valued cost functions provide essential measures for finding
discriminant functions of minimum risk binary classification systems—subject
to random vectors.

We have discovered that the inverse problem of the binary classification of
random vectors is directly related to the forward problem of the binary
classification of random vectors by fundamental laws of binary classification
that discriminant functions—of minimum risk binary classification systems,
subject to random vectors—are subject to. We have also discovered that the
theoretical model of a minimum risk binary classification system expresses
fundamental laws of binary classification, whereas the applied model of a
minimum risk binary classification system explains and executes these laws.

More generally, we have discovered that resolving the bias and variance
dilemma can be regarded as a system identification problem, so that the
overall statistical structure and behavior and properties of a system being
modeled are determined by transforming a collection of observations into a
data-driven mathematical model that represents fundamental aspects of the
system.

We have also discovered that the data-driven mathematical model executes
precise mathematical conditions that _statistically_ _pre-wire_ important
_generalizations_ within the _target function_ of the system—so that the
target function generalizes and thereby _extrapolates_ in a significant
manner.

Thereby, we have discovered that the statistical structure of a target
function is essential for its functionality—which includes the ability of the
target function to generalize.

We have discovered that formulating and solving certain system identification
problems involves determining how and why a particular system locates a point
of equilibrium—so that the energy exhibited by the system is minimized in such
a manner that the system satisfies a state of equilibrium—at which point the
structure and behavior and properties of the system exhibit the maximum amount
of stability.

Correspondingly, we have discovered that formulating and solving certain
system identification problems involves finding a suitable equivalent
representation of a given system—that requires finding a suitable statistical
representation for the transformed basis of an intrinsic coordinate system of
the given system.

We have discovered that resolving the bias and variance dilemma for certain
data-driven mathematical modeling problems involves the identification of
certain processes—which execute certain methods—that represent the solution
for finding the target function of a data-driven mathematical system, at which
point the problem being considered is transformed into a feasible one.

We have discovered that determining the generalization behavior of certain
machine learning algorithms involves solving a system identification problem,
so that the overall statistical structure and behavior and properties of a
system are determined by transforming a collection of observations into a
data-driven mathematical model that represents fundamental aspects of the
system.

As a final point, we have discovered that definite claims regarding the
generalization performance that is exhibited by certain machine learning
algorithms requires a proof, from first principles, which demonstrates that a
target function of a system has a certain statistical structure, along with a
constructive proof—which demonstrates that the machine learning algorithm is
aligned with the statistical structure of the target function of the system.

##  26 Acknowledgments

The author is indebted to Oscar Gonzalez and Garry Jacyna. The counsels of
Oscar Gonzalez and Garry Jacyna motivated the author to learn from both
breakthroughs and mistakes—and thereby persevere—and stay the course. The
author’s master’s thesis [66] was the primary impetus for this work. The
counsel of Oscar Gonzalez motivated the trailblazer within the author. Some of
the material in this treatise includes portions of the author’s Ph.D.
dissertation [67]. Initial parts of this work would not have occurred without
the support of Garry Jacyna. The counsel of Garry Jacyna sustained the
trailblazer within the author—and also enabled the author to successfully
navigate the Ph.D. pipeline.

## References

  * Allen and Mills [2004] R.L. Allen and D. Mills.  _Signal Analysis: Time, Frequency, Scale, and Structure_.  Wiley, 2004.  ISBN 9780471660361.
  * Aronszajn [1950] N. Aronszajn.  Theory of reproducing kernels.  _Transactions of the American Mathematical Society_ , pages 337–404, 1950.
  * Bellman et al. [1961] R. Bellman, R.E. Bellman, and Karreman Mathematics Research Collection.  _Adaptive Control Processes: A Guided Tour_.  Princeton Legacy Library. Princeton University Press, 1961.  ISBN 9780691079011.
  * Bennett and Campbell [2000] K. P. Bennett and C. Campbell.  Support vector machines: Hype or hallelujah?  _SIGKDD Explorations_ , 2(2):1–13, 2000.
  * Bernardo and Smith [2009] J.M. Bernardo and A.F.M. Smith.  _Bayesian Theory_.  Wiley Series in Probability and Statistics. Wiley, 2009.  ISBN 9780470317716.
  * Berrar [2018] Daniel Berrar.  Bayes theorem and naive bayes classifier.  _Encyclopedia of Bioinformatics and Computational Biology: ABC of Bioinformatics; Elsevier Science Publisher: Amsterdam, The Netherlands_ , pages 403–412, 2018.
  * Bishop [2006] Christopher M. Bishop.  _Pattern Recognition and Machine Learning_.  Springer, 2006.
  * Blakeslee and Blakeslee [2008] S. Blakeslee and M. Blakeslee.  _The Body Has a Mind of Its Own: How Body Maps in Your Brain Help You Do (Almost) Everything Better_.  Random House Publishing Group, 2008.  ISBN 9780812975277.
  * Boser et al. [1992] B. Boser, I. Guyon, and V. Vapnik.  A training algorithm for optimal margin classifiers.  In _Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory_ , pages 144–152. ACM Press, 1992.
  * Braithwaite [2007] Jason J Braithwaite.  Occam s razor: The principle of parsimony.  _Academia. edu_ , 2007.
  * Breiman [1991] L. Breiman.  Statistical modeling: The two cultures.  _Statistical Science_ , 16(3):199–231, 1991.
  * Burges [1998] C. Burges.  A tutorial on support vector machines for pattern recognition.  _Data Mining and Knowledge Discovery_ , 2:121–167, 1998\.
  * Byun and Lee [2002] H. Byun and S. Lee.  Applications of support vector machines for pattern recognition: A survey.  _LNCS_ , 2388:213–236, 2002.
  * Cortes and Vapnik [1995] C. Cortes and V. Vapnik.  Support-vector networks.  _Machine Learning_ , 20(3):273–297, 1995.
  * Cousins [1995] Robert D Cousins.  Why isn t every physicist a bayesian?  _American Journal of Physics_ , 63(5):398–410, 1995.
  * Cowan [2007] Glen Cowan.  Data analysis: frequently bayesian.  _Phys. Today_ , 60(4):82–83, 2007.
  * Cristianini and Shawe-Taylor [2000] N. Cristianini and J. Shawe-Taylor.  _An Introduction to Support Vector Machines and Other Kernel-based Learning Methods_.  Cambridge University Press, 2000.
  * Drake [1967] A.W. Drake.  _Fundamentals of Applied Probability Theory_.  McGraw-Hill classic textbook reissue series. McGraw-Hill, 1967.  ISBN 9780070178151.
  * Duda et al. [2001] R. Duda, P. Hart, and D. Stork.  _Pattern Classification_.  John Wiley and Sons, 2001.
  * Efron [2005] Bradley Efron.  _Modern science and the Bayesian-frequentist controversy_.  Division of Biostatistics, Stanford University, 2005.
  * Eisenhart [1939] L. Eisenhart.  _Coordinate Geometry_.  Dover Publications, 1939.
  * Eitrich and Lang [2006] T. Eitrich and B. Lang.  Efficient optimization of support vector machine learning parameters for unbalanced datasets.  _J COMPUT APPL MATH_ , 196(2):425–436, 2006.
  * Engl et al. [2000] W. H. Engl, M. Hanke, and A. Neubauer.  _Regularization of Inverse Problems_.  Kluwer Academic Publishers, 2000.
  * Fletcher [2000] R. Fletcher.  _Practical Methods of Optimization_.  Wiley, 2000.
  * Fowler [1980] R.H. Fowler.  _Statistical Mechanics: The Theory of the Properties of Matter in Equilibrium_.  Cambridge University Press, 1980.  ISBN 9780521093774.
  * Fukunaga [1990] K. Fukunaga.  _Introduction to Statistical Pattern Recognition_.  Academic Press, 1990.
  * Geman et al. [1992] S. Geman, E. Bienenstock, and R. Doursat.  Neural networks and the bias/variance dilemma.  _Neural Computation_ , 4:1–58, 1992.
  * Gibbs [2010] J.W. Gibbs.  _Elementary Principles in Statistical Mechanics: Developed with Especial Reference to the Rational Foundation of Thermodynamics_.  Cambridge Library Collection - Mathematics. Cambridge University Press, 2010.  ISBN 9781108017022.
  * Groetsch [1984] C. Groetsch.  _The Theory of Tikhonov Regularization for Fredholm Equations of the First Kind_.  Pitman Advanced Publishing Group, 1984.
  * Groetsch [1993] C. Groetsch.  _Inverse Problems in the Mathematical Sciences_.  Vieweg, 1993.
  * Hansen [1998] P. C. Hansen.  _Rank-Deficient and Discrete Ill-Posed Problems_.  SIAM, 1998.
  * Hewson [2009] S. Hewson.  _A Mathematical Bridge: An Intuitive Journey in Higher Mathematics_.  World Scientific, 2009.
  * Hilbert and Cohn-Vossen [1952] D. Hilbert and S. Cohn-Vossen.  _Geometry and the Imagination_.  Chelsea Publishing Company, 1952.
  * Hill [1956] T.L. Hill.  _Statistical Mechanics: Principles and Selected Applications_.  Dover Books on Physics. McGraw-Hill, 1956.  ISBN 9780486653907.
  * Hippenstiel [2017] R.D. Hippenstiel.  _Detection Theory: Applications and Digital Signal Processing_.  CRC Press, 2017.  ISBN 9781420042047.
  * Jackson [2000] E.A. Jackson.  _Equilibrium Statistical Mechanics_.  Dover books on physics. Dover Publications, 2000.  ISBN 9780486411859.
  * Kay [1998] S.M. Kay.  _Fundamentals of Statistical Signal Processing -: Detection Theory_.  Fundamentals of Statistical Signal Processing. PTR Prentice-Hall, 1998\.  ISBN 9780135041352.
  * Keener [2000] J. Keener.  _Principles of Applied Mathematics, Transformation and Approximation_.  Perseus Books, 2000.
  * Lathi [1998] B. P. Lathi.  _Signal Processing and Linear Systems_.  Berkley-Cambridge, 1998.
  * Lay [2006] D. C. Lay.  _Linear Algebra and Its Applications_.  Addison Wesley, 2006.
  * Liang et al. [2011] Y. Liang, Q. Xu, H. Li, and D. Cao.  _Support Vector Machines and Their Application in Chemistry and Biotechnology_.  CRC Press, 2011.
  * Linz [1979] P. Linz.  _Theoretical Numerical Analysis_.  Dover Publications, 1979.
  * Linz and Wang [2003] P. Linz and R. Wang.  _Exploring Numerical Methods_.  Jones and Bartlett Publishers, 2003.
  * Lipton [2011] B.H. Lipton.  _The Biology of Belief: Unleashing the Power of Consciousness, Matter & Miracles_.  Hay House, 2011.
  * Ljung [1998a] L. Ljung.  _System Identification: Theory for the User_.  Pearson Education, 1998a.  ISBN 9780132440530.
  * Ljung [1998b] Lennart Ljung.  System identification.  In _Signal analysis and prediction_ , pages 163–173. Springer, 1998b.
  * Luenberger [1969] D. Luenberger.  _Optimization by Vector Space Methods_.  John Wiley and Sons, 1969.
  * Luenberger [2003] D. Luenberger.  _Linear and Nonlinear Programming_.  Kluwer Academic Publishers, 2003.
  * Mate [2011] G. Mate.  _When the Body Says No: Understanding the Stress-Disease Connection_.  John Wiley & Sons, 2011.  ISBN 9780470923351.
  * McGrayne [2011] S.B. McGrayne.  _The Theory That Would Not Die: How Bayes’ Rule Cracked the Enigma Code, Hunted Down Russian Submarines, & Emerged Triumphant from Two Centuries of C_.  Matematicas (E-libro). Yale University Press, 2011.  ISBN 9780300175097.
  * Melsa and Cohn [1978] J.L. Melsa and D.L. Cohn.  _Decision and Estimation Theory_.  International student edition. McGraw-Hill Kogakusha, 1978.  ISBN 9780070664050.
  * Meyer [2000] C. Meyer.  _Matrix Analysis and Applied Linear Algebra_.  SIAM, 2000.
  * Mitchell [1997] T.M. Mitchell.  _Machine Learning_.  McGraw-Hill International Editions. McGraw-Hill, 1997.  ISBN 9780071154673.
  * Nash and Sofer [1996] S. Nash and A. Sofer.  _Linear and Nonlinear Programming_.  McGraw-Hill Company, 1996.
  * Naylor and Sell [1971] A. Naylor and G. Sell.  _Linear Operator Theory in Engineering and Science_.  Holt Rinehart and Winston, 1971.
  * Newton et al. [1999] I. Newton, I.B. Cohen, A. Whitman, and J. Budenz.  _The Principia: Mathematical Principles of Natural Philosophy_.  University of California Press, 1999.  ISBN 9780520290884.
  * Nichols [1893] E. Nichols.  _Analytic Geometry_.  D. C. Heath and Company, 1893.
  * Papoulis and Pillai [2002] A. Papoulis and S.U. Pillai.  _Probability, Random Variables, and Stochastic Processes_.  McGraw-Hill series in electrical engineering: Communications and signal processing. McGraw-Hill, 2002.  ISBN 9780070486584.
  * Parzen [1962] E. Parzen.  _Stochastic Processes_.  Holden-Day series in probability and statistics. Holden-Day, 1962.  ISBN 9780816266647.
  * Parzen [1960] Emanuel Parzen.  _Modern Probability Theory and Its Applications_.  Wiley-Interscience, 1960.
  * Pert et al. [1997] C.B. Pert, D. Chopra, and USA) Candace Pert (Biochemikerin.  _Molecules of Emotion: Why You Feel the Way You Feel_.  Touchstone Book. Scribner, 1997.
  * Poor [1994] H. Poor.  _An Introduction to Signal Detection and Estimation_.  Springer, 1994.
  * Rapport and Wright [1963] Samuel Rapport and Helen Wright.  _Mathematics_.  New York University Press, 1963.
  * Ravella [2022] S. Ravella.  _A Silent Fire: The Story of Inflammation, Diet, and Disease_.  W. W. Norton, 2022.  ISBN 9780393541915.
  * Reeves and Jacyna [2011] D. M. Reeves and G. M. Jacyna.  Support vector machine regularization.  _WIREs Computational Statistics_ , 3:204–215, 2011.
  * Reeves [1995] Denise M. Reeves.  Generalization metrics for neural modeling applications in system identification.  Master’s thesis, Old Dominion University, 1995.
  * Reeves [2009] Denise M. Reeves.  _Properly Specified Functional Mappings and Support Vector Learning Machines_.  PhD thesis, George Mason University, 2009.
  * Reeves [2015] Denise M. Reeves.  Resolving the geometric locus dilemma for support vector learning machines.  _arXiv preprint arXiv:1511.05102_ , 2015.
  * Reeves [2018] Denise M. Reeves.  Design of data-driven mathematical laws for optimal statistical classification systems.  _arXiv preprint arXiv:1612.03902_ , 2018.
  * Rice [1995] J.A. Rice.  _Mathematical Statistics and Data Analysis_.  Duxbury advanced series. Duxbury Press, 1995.  ISBN 9780534209346.
  * Ross [1998] Sheldon M. Ross.  _A First Course in Probability_.  Prentice-Hall, 1998.
  * Ross [2007] S.M. Ross.  _Introduction to Probability Models_.  Introduction to Probability Models. Elsevier Science, 2007.  ISBN 9780125980623.
  * Scholkopf and Smola [2002] B. Scholkopf and A. Smola.  _Learning with Kernels_.  MIT Press, 2002.
  * Small and McLeish [1994] Christopher G. Small and Don L. McLeish.  _Hilbert Space Methods in Probability and Statistical Inference_.  Wiley-Interscience, 1994.
  * Srinath et al. [1996] M. Srinath, P. Rajasekaran, and R. Viswanathan.  _Statistical Signal Processing with Applications_.  Prentice Hall, 1996.
  * Stark and Woods [1994] H. Stark and J.W. Woods.  _Probability, Random Processes, and Estimation Theory for Engineers_.  Industrial and Systems Engineering. Prentice Hall, 1994.  ISBN 9780137287918.
  * Sternberg [2001] E.M. Sternberg.  _The Balance Within: The Science Connecting Health and Emotions_.  Henry Holt and Company, 2001.
  * Strang [1986] G. Strang.  _Introduction to Applied Mathematics_.  Wellesley-Cambridge Press, 1986.  ISBN 9780961408800.
  * Strang [2007] G. Strang.  _Computational Science and Engineering_.  Wellesley-Cambridge Press, 2007.  ISBN 9780961408817.
  * Sundaram [1996] Rangarajan Sundaram.  _A First Course in Optimization Theory_.  Cambridge University Press, 1996.
  * Tanner and Allen [1898] Tanner and Allen.  _Analytic Geometry_.  American Book Company, 1898.
  * Thomas and Finney [1995] G.B. Thomas and R.L. Finney.  _Calculus and Analytic Geometry_.  Addison-Wesley world student series. Addison-Wesley Publishing Company, 1995.  ISBN 9780201531824.
  * Tiemens et al. [2020] Bea Tiemens, Renée Wagenvoorde, and Cilia Witteman.  Why every clinician should know bayes’ rule.  _Health Professions Education_ , 6(3):320–324, 2020.  ISSN 2452-3011.
  * van der Kolk [2014] B. van der Kolk.  _The Body Keeps the Score: Mind, Brain and Body in the Transformation of Trauma_.  Penguin Books Limited, 2014.  ISBN 9780141978628.
  * VanTrees [1968] H. VanTrees.  _Detection, Estimation, and Modulation Theory: Part I_.  John Wiley and Sons, 1968.
  * Wahba [1987] G. Wahba.  _Inverse and Ill-Posed Problems_.  Academic Press, 1987.
  * Wolpert [2001] David H. Wolpert.  The supervised learning no-free-lunch theorems.  In _In Proc. 6th Online World Conference on Soft Computing in Industrial Applications_ , pages 25–42, 2001.
  * Wolpert [2020] David H. Wolpert.  What is important about the no free lunch theorems?  _arXiv preprint arXiv:2007.10928_ , 2020.
  * Zwillinger [1996] Daniel Zwillinger.  _Standard Mathematical Tables And Formulae_.  CRC Press, 1996.