#  $\ell_{p}$ Slack Norm Support Vector Data Description

Shervin Rahimzadeh Arashloo111Corresponding author: Shervin Rahimzadeh
Arashloo.
E-mail<EMAIL_ADDRESS>
###### Abstract

The support vector data description (SVDD) approach serves as a de facto
standard for one-class classification where the learning task entails
inferring the smallest hyper-sphere to enclose target objects while linearly
penalising any errors/slacks via an $\ell_{1}$-norm penalty term. In this
study, we generalise this modelling formalism to a general $\ell_{p}$-norm
($p\geq 1$) slack penalty function. By virtue of an $\ell_{p}$ slack norm, the
proposed approach enables formulating a non-linear cost function with respect
to slacks. From a dual problem perspective, the proposed method introduces a
sparsity-inducing dual norm into the objective function, and thus, possesses a
higher capacity to tune into the inherent sparsity of the problem for enhanced
descriptive capability.

A theoretical analysis based on Rademacher complexities characterises the
generalisation performance of the proposed approach in terms of parameter $p$
while the experimental results on several datasets confirm the merits of the
proposed method compared to other alternatives.

###### keywords:

One-class classification, kernel methods , support vector data description ,
$\ell_{p}$ slack norm

##  1 Introduction

One-class classification (OCC) addresses the problem of recognising patterns
that adhere to a specific condition presumed as normal, and identifying them
from any other object violating the normality criterion. OCC stands apart from
the conventional two-/multi-class classification paradigm in that it primarily
uses observations from a single, very often the target class for training.
One-class classification acts as an essential building block in a diverse
range of practical systems including presentation attack detection in
biometrics [1], health care [2], audio or video surveillance [3, 4], intrusion
detection [5], social network [6], safety-critical systems [7], fraud
detection [8], insurance [9], etc.

As with many other machine learning problems, state-of-the-art OCC algorithms
are built on the premise of deep learning methodology [10, 11, 12] using
massive labelled datasets, typically containing millions of samples. Although
deep structures have led to breakthroughs in one-class learning and
classification, their reliance on huge sets of data may pose certain
limitations in practice. In this context, collecting sufficiently large sets
of training observations for certain applications can be a challenge,
hindering a full exploitation of the expressive capacity of deep networks.
Even if sufficient data is gathered, labelling such huge amounts of data may
be a bigger challenge. Whilst crowd-sourcing may be considered as an
applicable strategy to label huge sets of data in some fields, for a variety
of different reasons including level of knowledge, data privacy, time required
to produce accurate labels, etc. it may not serve as a viable option in the
domains such as defence, security, healthcare. Although certain techniques
such as active learning [13] or learning with privileged information [14] may
be instrumental in reducing the quantity of necessary labelling/labelled data,
they still demand the time and domain expertise of a human operator. In the
absence of large training sets required by deep nets, and specifically for
small to moderate-sized datasets containing hundreds or thousands of training
samples, kernel-based methods [15] offer a very promising methodology of
classification. Moreover, unlike deep networks that incorporate many
heuristics with regards to their structure and the corresponding
(hyper)parameters, kernel methods are based on solid foundations and are
characterised by strong bases in optimisation and statistical learning theory
[16].

The support vector data description (SVDD) approach [17] which is proposed as
an adaptation of support vector machines to the one-class setting, presents a
very popular kernel-based method for one-class classification. Although
designed for one-class setting, the SVDD approach does not require the
training data to be exclusively and purely normal/positive which can be
regarded as a quite appealing property in practical applications where the
data is very often contaminated with noise and outliers. Furthermore, it
provides an intuitive geometric characterisation of a predominantly positive
dataset without making any specific assumption regarding the underlying
distribution. Moreover, the SVDD decision making process entails computing a
simple distance between the centre of the target class and a test observation
to label it as either normal (i.e. positive/target) or as anomaly (i.e.
negative/outlier). And finally, when large sets of training data are
available, the SVDD method may be extended to a deep structure to directly
learn features from the data for improved performance [11]. These properties
make SVDD a highly favoured method of practice in a variety of one-class
classification applications where it serves as one of the most widely used
techniques, if not the most.

The underlying idea in the SVDD approach is to determine the smallest hyper-
sphere enclosing the data. While its hard-margin formulation requires all
positive target data to be strictly encapsulated within the inferred hyper-
spherical boundary, in practical situations, a dataset may incorporate
noisy/outlier samples. In the soft margin SVDD approach, in order to take into
account the possibility of a contaminated dataset and improve the
generalisation capability of the model, the distance from each training object
to the centre of the hyper-sphere need not be strictly smaller than the radius
but larger distances are penalised. In order to encode and penalise violations
from the hyper-spherical decision boundary, in the soft margin variant, non-
negative slack variables measuring the extent of violation of each object from
the decision boundary are introduced. The optimisation problem is then
modified to reflect such violations and penalise an $\ell_{1}$-norm term on
the slacks. In other words, the conventional SVDD method, and also the
standard two-class SVM classifier [15] are founded on the idea of minimising
an $\ell_{1}$-norm risk over the set of non-negative slack variables. In the
context of two-class classification, very recently [18], the classical
$\ell_{1}$-norm penalty term in the SVM formulation has been revisited to
consider two alternative slack penalties defined by the $\ell_{2}$\- and the
$\ell_{\infty}$-norms to formulate new SVM algorithms. A reformulation of the
standard two-class SVM to the $\ell_{2}$ and $\ell_{\infty}$ penalty terms has
been verified to improve the classification performance, sometimes
significantly [18].

In this work, we study the merits of different norm risks for ”one-class”
classification in the context of the SVDD approach. For this purpose, we
consider a general $\ell_{p}$-norm ($p\geq 1$) slack penalty term where $p$
serves as a free parameter of the algorithm. As such, while in the standard
SVDD method the slacks are penalised linearly, by introducing a $p$-norm
function, non-linear cost functions of the slacks may be optimised where the
degree of the non-linearity (i.e. $p$) may be tuned on the data. The
reflection of the $\ell_{p}$ slack penalty term onto the dual space
formulation of the corresponding optimisation problem turns out to be a dual
$\ell_{q}$-norm ($q=\frac{p}{p-1}$) cost on the dual variables, thus,
providing the method the capability to tune into the inherent sparsity of the
problem.

###  1.1 Contributions

The major contributions of the current study may be summarised as listed
below.

  * •

We generalise the SVDD formulation from an $\ell_{1}$ to an $\ell_{p}$ slack
norm penalty function and illustrate that the proposed generalisation may lead
to significant improvements in the performance of the algorithm;

  * •

We extend the proposed $\ell_{p}$-norm formulation from a pure one-class
setting to the training scenario where labelled negative objects are also
available and illustrate the merits offered by the proposed extension;

  * •

Based on Rademacher complexities, we theoretically study the generalisation
performance of the proposed $\ell_{p}$ slack norm approach and derive bounds
on its error;

  * •

And we carry out an experimental evaluation of the proposed method on multiple
OCC datasets and provide a comparison to the original SVDD method and its
different variants, as well as other OCC techniques from the literature.

###  1.2 Organisation

The remainder of the paper is structured as follows. In Section 2, the
relevant literature with a particular emphasis on different variants of the
SVDD formalism is reviewed. In Section 3, once a short overview of the support
vector data description (SVDD) approach [17] is given, we present our proposed
$\ell_{p}$ slack norm SVDD approach for the pure one-class setting and then
derive its extension for labelled negative training observations. Section 4
studies the generalisation error bound of the proposed approach based on
Rademacher complexities. We present and analyse the results of an experimental
evaluation of the proposed method in Section 5 where possible extensions of
the proposed approach are also discussed. Finally, Section 6 concludes the
paper.

##  2 Prior work

Although different categorisations of OCC methods exist in different studies
[19, 20, 21, 22, 17], the one-class classification techniques may be roughly
identified as either generative or non-generative [23], the latter best
represented by discriminative approaches. While in the generative techniques,
the objective is to model the underlying generative process of the data, the
discriminative methods try to directly partition the observation space into
different regions for classification. Discriminative approaches tend to yield
better performance in practice since they try to explicitly solve the OCC
problem without attempting to solve an intermediate and more general task of
inferring the underlying distribution or generative process [16].

Generative OCC approaches encompass the methods that try to estimate the
underlying distribution using, for example, Parzen windowing [24], Gaussian
distribution modelling [25], or those which use a mixture of distributions
[26, 27]. A different sub-category of generative approaches include methods
that for decision making use the residual of reconstructing a test sample with
respect to a hypothesised model, some instances of which are the kernel
principal component analysis (KPCA) and its variants [28, 29], or the
autoencoder-based techniques [30].

Discriminative methods constitute a strong alternative to the generative one-
class learners. As an instance, based on a variant of the Fisher
classification principle, the kernel null space method [31] tries to map
positive objects onto a single point in a discriminative subspace, obtaining
very competitive results compared to some other alternatives [32]. Another
successful discriminative one-class method focuses on the use of Gaussian
Process (GP) priors [33] trying to directly infer the a posteriori class
probability of the target class. A further example of discriminative one-class
learners is that of the nearest neighbour-based approaches [34] where the
normality of an object is decided based on its immediate neighbours. Among
others, a widely applied discriminative one-class classification method is
that of support vector data description (SVDD) approach [17] that tries to
estimate the smallest volume surrounding the positive objects. In the case of
the existence of labelled negative training objects, the decision boundary is
refined by requiring the negative objects to lie outside the hyper-spherical
boundary. The soft version of this approach allows the positive and negative
(if any) training objects to violate the boundary criterion but subject to a
linear penalty on the extent of the violation (called slack) where a parameter
controls the trade-off between the volume and such errors in the objective
function. Due to its success in data description and its intuitive geometrical
interpretation and the ability to benefit from a kernel-based representation,
the SVDD approach serves as a widely used technique in the OCC literature,
motivating many subsequent research. As an instance, in [35], based on the
observation that the SVDD centre and the volume are sensitive to the parameter
controlling the trade-off between the errors (slacks) and the volume, a method
called GL-SVDD is proposed where local and global probability densities are
used to derive sample-adaptive errors via associating weights to the slacks
corresponding to different objects. In [36], a different sample-specific
weighting approach (P-SVDD) based on the position of the feature space image
is proposed to adaptively regularise the complexity of the SVDD sphere. Other
work [37] (DW-SVDD) considers re-weighting sample errors in the objective
function by utilising the relative density of each object to the density
distribution of normal samples. The authors in [38] define a density-based
distance between a sample point and the centre of the hyper-sphere to adjust
the constraint set of the SVDD optimisation problem by re-weighting training
objects. The work in [39], considers a different linear sample weighting
scheme in the SVDD objective function by introducing the cut-off distance-
based local density of objects. The work in [40] introduces a margin parameter
to maximise the margin between the hyper-sphere and the non-target objects in
an SVDD formulation and directly optimises the margin. The Euclidean distance
($\ell_{2}$ distance) employed in the widely used Gaussian kernel function is
reassessed in [5] to see if other distances in the Gaussian kernel function
may provide performance advantages. Apart from the research focused on
improving the performance of the SVDD method in a one-class setting, there
also exist other studies where the SVDD approach is generalised to two [41],
or to multiple classes [42].

Considering the body of work discussed above, one observes that the majority
of the existing studies tries to modify the slack error term by introducing an
adaptive weighting for each data sample based on different cues. Clearly, a
simple linear weighting scheme does change the linearity of the objective
function with respect to the slacks. The exceptions to the studies above are
the work in [43, 44] where instead of an $\ell_{1}$-norm penalty, an
$\ell_{2}$-norm penalty is considered over the slacks. As will be demonstrated
in the subsequent sections, an $\ell_{2}$ slack penalty may not always yield
an optimal performance for data description. The current study is a
generalisation of the existing SVDD formulations as it considers an $\ell_{p}$
($p\geq 1$) slack norm penalty where $p$ serves as a free parameter of the
algorithm allowing for different non-linear penalties to be optimised w.r.t.
slacks while at the same time providing the opportunity to tune into the
inherent sparsity characteristics of the data.

##  3 Methodology

In this section, first, we briefly review the SVDD method [17] and then
present the proposed approach.

###  3.1 Preliminaries

The Support Vector Data Description (SVDD) approach [17] tries to estimate the
smallest hyperspherical volume that encloses normal/target data in some pre-
determined feature space. As a hypersphere is characterised by its centre
$\mathcal{O}$ and its radius $R>0$, the learning problem in the SVDD method is
defined as minimising the radius while requiring the hypersphere to
encapsulate all normal objects $\mathbf{x}_{i}$’s, that is

$\displaystyle\min_{R,\mathcal{O}}E(R,\mathcal{O})=R^{2}$
$\displaystyle\text{s.t.
}\left\lVert\mathbf{x}_{i}-\mathcal{O}\right\rVert_{2}^{2}\leq R^{2},\text{
}\forall i$ (1)

In practice, however, the training data might be contaminated with noise and
outliers. In order to handle possible outliers in the training set and derive
a solution with a better generalisation capability, the objective function in
the SVDD method is modified so that the distance from the centre $\mathcal{O}$
to each training observation $\mathbf{x}_{i}$ need not be strictly smaller
than $R$, rather, larger distances are penalised. For this purpose, using non-
negative slack variables $\zeta_{i}$’s, the SVDD optimisation task is modified
as

$\displaystyle\min_{R,\mathcal{O},\boldsymbol{\zeta}}E(R,\mathcal{O},\boldsymbol{\zeta})=R^{2}+c\sum_{i}\zeta_{i}$
$\displaystyle\text{s.t.
}\left\lVert\mathbf{x}_{i}-\mathcal{O}\right\rVert_{2}^{2}\leq
R^{2}+\zeta_{i},\text{ }\zeta_{i}\geq 0,\text{ }\forall i$ (2)

where $\boldsymbol{\zeta}$ denotes a vector collection of $\zeta_{i}$’s and
the trade-off between the sum of errors (i.e. $\zeta_{i}$’s) and the squared
radius is controlled using parameter $c$. The optimisation problem above
corresponds to the case where only normal samples (and possibly a minority of
noisy objects) are presumed to exist in the training set. When labelled
negative training objects are also available, the learning problem in the SVDD
method is modified to enforce positive samples to lie within the hyper-sphere
while negative samples are encouraged to fall outside its boundary.

The SVDD objective function in Eq. 2 depends on an $\ell_{1}$-norm of the
slack variables as
$\sum_{i}\zeta_{i}=\left\lVert\boldsymbol{\zeta}\right\rVert_{1}$, and
consequently, all errors/slacks are penalised linearly. Although penalising
all errors linearly in their magnitudes is a plausible option, it is by no
means the only possibly. An an instance, a different alternative may be to
penalise only the maximum error/slack which can be achieved by incorporating
an $\ell_{\infty}$-norm on the slacks as
$\max_{i}\zeta_{i}=\left\lVert\boldsymbol{\zeta}\right\rVert_{\infty}$. Any
other penalty which would lie between penalising all the slacks linearly and
penalising only the maximum error may then be characterised using a general
$\ell_{p}$-norm on the errors, i.e. via
$\sum_{i}\zeta_{i}^{p}=\left\lVert\boldsymbol{\zeta}\right\rVert_{p}^{p}$. In
particular, introducing a variable norm parameter $p$ opens the door to
consider non-linear penalties on the errors compared with the original SVDD
method which is limited to a linear penalty on the slacks. From a dual problem
viewpoint, introducing an $\ell_{p}$ norm penalty on the slacks translates
into sparsity inducing dual norms on the dual variables which provides the
opportunity to better consider the intrinsic sparsity of the problem. As such,
in the proposed approach, we generalise the SVDD error function using an
$\ell_{p}$-norm function of slacks, discussed next.

###  3.2 $\ell_{p}$ slack norm SVDD

By replacing the $\ell_{1}$-norm term on the slack variables in Eq. 2 with an
$\ell_{p}$-norm, the optimisation problem in the proposed approach is defined
as

$\displaystyle\min_{R,\mathcal{O},\boldsymbol{\zeta}}E(R,\mathcal{O})=R^{2}+c\sum_{i}\zeta_{i}^{p}$
$\displaystyle\text{s.t.
}\left\lVert\mathbf{x}_{i}-\mathcal{O}\right\rVert_{2}^{2}\leq
R^{2}+\zeta_{i},\text{ }\zeta_{i}\geq 0,\text{ }\forall i$ (3)

In order to solve the optimisation problem above, the Lagrangian is formed as

$\displaystyle\mathcal{L}=R^{2}+c\sum_{i}\zeta_{i}^{p}-\sum_{i}\alpha_{i}[R^{2}+\zeta_{i}-(\left\lVert\mathbf{x}_{i}\right\rVert_{2}^{2}-2\mathcal{O}^{\top}\mathbf{x}_{i}+\left\lVert\mathcal{O}\right\rVert_{2}^{2})]-\sum_{i}\gamma_{i}\zeta_{i}$
(4)

where $\alpha_{i}$’s and $\gamma_{i}$’s are non-negative Lagrange multipliers.
In order to derive the dual function, the Lagrangian should be minimised with
respect to the primal variables $R$, $\mathcal{O}$, $\zeta_{i}$. Setting the
partial derivatives of $\mathcal{L}$ w.r.t. $R$, $\mathcal{O}$, and
$\zeta_{i}$ to zero yields

$\displaystyle\frac{\partial\mathcal{L}}{\partial R}=0\hskip
10.0pt\Rightarrow\hskip 10.0pt\sum_{i}\alpha_{i}=1$ (5a)
$\displaystyle\frac{\partial\mathcal{L}}{\partial\mathcal{O}}=0\hskip
10.0pt\Rightarrow\hskip 10.0pt\mathcal{O}=\sum_{i}\alpha_{i}\mathbf{x}_{i}$
(5b) $\displaystyle\frac{\partial\mathcal{L}}{\partial\zeta_{i}}=0\hskip
10.0pt\Rightarrow\zeta_{i}=(\frac{\alpha_{i}+\gamma_{i}}{cp})^{\frac{1}{p-1}}$
(5c)

Substituting the relations above into $\mathcal{L}$, the Lagrangian is
obtained as

$\displaystyle\mathcal{L}=(cp)^{\frac{-1}{p-1}}(1/p-1)\left\lVert\boldsymbol{\alpha}+\boldsymbol{\gamma}\right\rVert_{p/(p-1)}^{p/(p-1)}+\sum_{i}\alpha_{i}\mathbf{x}_{i}^{\top}\mathbf{x}_{i}-\sum_{i}\sum_{j}\alpha_{i}\alpha_{j}\mathbf{x}_{i}^{\top}\mathbf{x}_{j}$
(6)

where $\boldsymbol{\alpha}$ and $\boldsymbol{\gamma}$ denote vector
collections of $\alpha_{i}$’s and $\gamma_{i}$’s. Furthermore, one can easily
check that the Slater’s condition is satisfied, and thus, the following
complementary conditions also hold at the optimum:

$\displaystyle\gamma_{i}\zeta_{i}=0,\forall i$ (7a)
$\displaystyle\alpha_{i}(R^{2}+\zeta_{i}-\left\lVert\mathbf{x}_{i}-\mathcal{O}\right\rVert_{2}^{2})=0,\forall
i$ (7b)

Using Eq. 5c and Eq. 7a, it must hold that
$\gamma_{i}(\frac{\alpha_{i}+\gamma_{i}}{cp})^{\frac{1}{p-1}}=0$. Since
$\alpha_{i}\geq 0$ and $\gamma_{i}\geq 0$, one concludes that $\gamma_{i}=0$,
$\forall i$. As a result, the Lagrangian in Eq. 6 would be simplified as

$\displaystyle\mathcal{L}=(cp)^{\frac{-1}{p-1}}(1/p-1)\left\lVert\boldsymbol{\alpha}\right\rVert_{p/(p-1)}^{p/(p-1)}+\sum_{i}\alpha_{i}\mathbf{x}_{i}^{\top}\mathbf{x}_{i}-\sum_{i}\sum_{j}\alpha_{i}\alpha_{j}\mathbf{x}_{i}^{\top}\mathbf{x}_{j}$
(8)

The dual problem entails maximising $\mathcal{L}$ in $\boldsymbol{\alpha}$:

$\displaystyle\max_{\boldsymbol{\alpha}}\hskip 10.0pt\mathcal{L}$
$\displaystyle\text{s.t. }\boldsymbol{\alpha}\geq
0,\left\lVert\boldsymbol{\alpha}\right\rVert_{1}=1$ (9)

Note that, for $p\geq 1$, we have $p/(p-1)\geq 1$, and consequently, the term
$\left\lVert\boldsymbol{\alpha}\right\rVert_{p/(p-1)}^{p/(p-1)}$ in the
Lagrangian is convex w.r.t. $\boldsymbol{\alpha}$. Note also that the other
terms in $\mathcal{L}$ are either linear or quadratic functions of
$\boldsymbol{\alpha}$, and hence, are convex while the constraints are affine.
As a result, the optimisation problem above is a convex optimisation task.

###  3.3 $\ell_{p}$ slack norm SVDD with negative samples

In the proposed $\ell_{p}$ slack norm approach, similar to the original SVDD
method [17], when labelled non-target/negative training observations are
available, they may be utilised to refine the description. In this case, as
opposed to the positive samples that should be enclosed within the
hypersphere, the non-target objects should lie outside its boundary. In what
follows, the normal/positive samples are indexed by $i$, $j$ and the negative
objects by $l$, $m$. In order to allow for possible errors in both the
positive and the negative training samples, slack variables $\zeta_{i}$’s and
$\zeta_{l}$’s are introduced. The optimisation problem when labelled negative
samples are available is then defined as

$\displaystyle\min_{R,\mathcal{O},\boldsymbol{\zeta}}E(R,\mathcal{O},\boldsymbol{\zeta})=R^{2}+c_{1}\sum_{i}\zeta_{i}^{p}+c_{2}\sum_{l}\zeta_{l}^{p}$
$\displaystyle\text{s.t.
}\left\lVert\mathbf{x}_{i}-\mathcal{O}\right\rVert_{2}^{2}\leq
R^{2}+\zeta_{i},\text{
}\left\lVert\mathbf{x}_{l}-\mathcal{O}\right\rVert_{2}^{2}\geq
R^{2}-\zeta_{l},\text{ }\zeta_{i}\geq 0,\text{ }\zeta_{l}\geq 0,\text{
}\forall i,l$ (10)

In the objective function above, while $c_{1}$ may be used to control the
fraction of positive training objects that fall outside the hypersphere
boundary, $c_{2}$ may be adjusted to regulate the fraction of negative
training samples that will lie within the hypersphere. By introducing Lagrange
multipliers $\alpha_{i}\geq 0$, $\alpha_{l}\geq 0$, $\gamma_{i}\geq 0$,
$\gamma_{l}\geq 0$, the Lagrangian of Eq. 10 is formed as

$\displaystyle\mathcal{L}=R^{2}+c_{1}\sum_{i}\zeta_{i}^{p}+c_{2}\sum_{l}\zeta_{l}^{p}-\sum_{i}\gamma_{i}\zeta_{i}-\sum_{l}\gamma_{l}\zeta_{l}$
$\displaystyle-\sum_{i}\alpha_{i}[R^{2}+\zeta_{i}-(\left\lVert\mathbf{x}_{i}\right\rVert_{2}^{2}-2\mathcal{O}^{\top}\mathbf{x}_{i}+\left\lVert\mathcal{O}\right\rVert_{2}^{2})]$
$\displaystyle-\sum_{l}\alpha_{l}[(\left\lVert\mathbf{x}_{l}\right\rVert_{2}^{2}-2\mathcal{O}^{\top}\mathbf{x}_{l}+\left\lVert\mathcal{O}\right\rVert_{2}^{2})-R^{2}+\zeta_{l}]$
(11)

In order to form the dual function, the Lagrangian should be minimised w.r.t.
$R$, $\mathcal{O}$, $\zeta_{i}$’s, and $\zeta_{l}$’s. Setting the partial
derivatives of $\mathcal{L}$ w.r.t. to $R$, $\mathcal{O}$, $\zeta_{i}$, and
$\zeta_{l}$ to zero yields

$\displaystyle\frac{\partial\mathcal{L}}{\partial R}=0\hskip
10.0pt\Rightarrow\hskip 10.0pt\sum_{i}\alpha_{i}-\sum_{l}\alpha_{l}=1$ (12a)
$\displaystyle\frac{\partial\mathcal{L}}{\partial\mathcal{O}}=0\hskip
10.0pt\Rightarrow\hskip
10.0pt\mathcal{O}=\sum_{i}\alpha_{i}\mathbf{x}_{i}-\sum_{l}\alpha_{l}\mathbf{x}_{l}$
(12b) $\displaystyle\frac{\partial\mathcal{L}}{\partial\zeta_{i}}=0\hskip
10.0pt\Rightarrow\zeta_{i}=(\frac{\alpha_{i}+\gamma_{i}}{c_{1}p})^{\frac{1}{p-1}}$
(12c) $\displaystyle\frac{\partial\mathcal{L}}{\partial\zeta_{l}}=0\hskip
10.0pt\Rightarrow\zeta_{l}=(\frac{\alpha_{l}+\gamma_{l}}{c_{2}p})^{\frac{1}{p-1}}$
(12d)

Resubstituting the relations above into Eq. 11 gives

$\displaystyle\mathcal{L}$
$\displaystyle=(c_{1}p)^{\frac{-1}{p-1}}(1/p-1)\left\lVert\boldsymbol{\alpha}_{T}+\boldsymbol{\gamma}_{T}\right\rVert_{p/(p-1)}^{p/(p-1)}+(c_{2}p)^{\frac{-1}{p-1}}(1/p-1)\left\lVert\boldsymbol{\alpha}_{N}+\boldsymbol{\gamma}_{N}\right\rVert_{p/(p-1)}^{p/(p-1)}$
$\displaystyle+\sum_{i}\alpha_{i}\mathbf{x}_{i}^{\top}\mathbf{x}_{i}-\sum_{l}\alpha_{l}\mathbf{x}_{l}^{\top}\mathbf{x}_{l}-\sum_{i}\sum_{j}\alpha_{i}\alpha_{j}\mathbf{x}_{i}^{\top}\mathbf{x}_{j}-\sum_{l}\sum_{m}\alpha_{l}\alpha_{m}\mathbf{x}_{l}^{\top}\mathbf{x}_{m}$
$\displaystyle+2\sum_{i}\sum_{l}\alpha_{i}\alpha_{l}\mathbf{x}_{i}^{\top}\mathbf{x}_{l}$
(13)

where $\boldsymbol{\alpha}_{T}$ and $\boldsymbol{\alpha}_{N}$ respectively
stand for vector collections of $\alpha_{i}$’s and $\alpha_{l}$’s. Similarly,
$\boldsymbol{\gamma}_{T}$ and $\boldsymbol{\gamma}_{N}$ denote vector
collections of $\gamma_{i}$’s and $\gamma_{l}$’s, respectively. Since the
Slater’s condition holds, the following complementary conditions are also
satisfied at the optimum:

$\displaystyle\gamma_{i}\zeta_{i}=0,\forall i$ (14a)
$\displaystyle\gamma_{l}\zeta_{l}=0,\forall l$ (14b)
$\displaystyle\alpha_{i}(R^{2}+\zeta_{i}-\left\lVert\mathbf{x}_{i}-\mathcal{O}\right\rVert_{2}^{2})=0,\forall
i$ (14c)
$\displaystyle\alpha_{l}(R^{2}-\zeta_{l}-\left\lVert\mathbf{x}_{l}-\mathcal{O}\right\rVert_{2}^{2})=0,\forall
l$ (14d)

Using Eqs. 12c and 14a, and also Eqs. 12d and 14b, one concludes that
$\gamma_{i}=0$, $\forall i$ and $\gamma_{l}=0$, $\forall l$. As a result, the
Lagrangian in Eq. 13 would be

$\displaystyle\mathcal{L}$
$\displaystyle=(c_{1}p)^{\frac{-1}{p-1}}(1/p-1)\left\lVert\boldsymbol{\alpha}_{T}\right\rVert_{p/(p-1)}^{p/(p-1)}+(c_{2}p)^{\frac{-1}{p-1}}(1/p-1)\left\lVert\boldsymbol{\alpha}_{N}\right\rVert_{p/(p-1)}^{p/(p-1)}$
$\displaystyle+\sum_{i}\alpha_{i}\mathbf{x}_{i}^{\top}\mathbf{x}_{i}-\sum_{l}\alpha_{l}\mathbf{x}_{l}^{\top}\mathbf{x}_{l}-\sum_{i}\sum_{j}\alpha_{i}\alpha_{j}\mathbf{x}_{i}^{\top}\mathbf{x}_{j}-\sum_{l}\sum_{m}\alpha_{l}\alpha_{m}\mathbf{x}_{l}^{\top}\mathbf{x}_{m}$
$\displaystyle+2\sum_{i}\sum_{l}\alpha_{i}\alpha_{l}\mathbf{x}_{i}^{\top}\mathbf{x}_{l}$
(15)

The dual problem then reads

$\displaystyle\max_{\boldsymbol{\alpha}_{T},\boldsymbol{\alpha}_{N}}\hskip
10.0pt\mathcal{L}$ s.t. $\displaystyle\boldsymbol{\alpha}_{T}\geq
0,\boldsymbol{\alpha}_{N}\geq 0,$
$\displaystyle\left\lVert\boldsymbol{\alpha}_{T}\right\rVert_{1}-\left\lVert\boldsymbol{\alpha}_{N}\right\rVert_{1}=1$
(16)

Since $p\geq 1$ leads to $p/(p-1)\geq 1$, the terms
$\left\lVert.\right\rVert_{p/(p-1)}^{p/(p-1)}$ in the Lagrangian are convex
while the remaining terms are either linear or quadratic functions and the
constraint sets are affine. Subsequently, the maximisation problem in Eq. 16
is convex.

###  3.4 Joint formulation

As discussed earlier, when only positive labelled training observations are
available, in the proposed approach one solves the optimisation problem in Eq.
9 with the Lagrangian given in Eq. 8. When in addition to the positive
training samples, labelled negative training objects are also available, the
problem to be solved is expressed as the optimisation task in Eq. 16 with the
corresponding Lagrangian given in Eq. 15. Although the optimisation tasks
corresponding to the pure positive case and that of the second scenario where
negative training samples are also available may appear different,
nevertheless, both optimisation problems can be expressed compactly using a
joint formulation as follows. Let us assume that vector $\mathbf{y}$
corresponds to the labels of training samples where for positive objects the
label is $+1$ while for any possible non-target training samples the
corresponding label is $-1$. Furthermore, suppose the Lagrange multipliers
associated with the negative and positive samples are all collected into a
single vector $\boldsymbol{\alpha}$. In order to reduce the clutter in the
formulation, let us further assume $q=p/(p-1)$,
$\bar{c_{1}}=\frac{1}{2}(c_{1}p)^{\frac{-1}{p-1}}(1-1/p)$ and
$\bar{c_{2}}=\frac{1}{2}(c_{2}p)^{\frac{-1}{p-1}}(1-1/p)$. With these
definitions, the Lagrangian in Eq. 15 may be expressed as

$\displaystyle\mathcal{L}=-\bar{c_{1}}\left\lVert\boldsymbol{\alpha}\odot(1+\mathbf{y})\right\rVert_{q}^{q}-\bar{c_{2}}\left\lVert\boldsymbol{\alpha}\odot(1-\mathbf{y})\right\rVert_{q}^{q}+\sum_{i}\alpha_{i}y_{i}\mathbf{x}_{i}^{\top}\mathbf{x}_{i}-\sum_{i,j}\alpha_{i}y_{i}\alpha_{j}y_{j}\mathbf{x}_{i}^{\top}\mathbf{x}_{j}$
(17)

where $\odot$ denotes hadamard/elementwise product. It may be easily verified
that when only positive training samples are available, the Lagrangian above
correctly recovers that of Eq. 8 while in the existence of labelled negative
training objects, it matches that of Eq. 15. As a result, in the proposed
approach, the generic optimisation problem to solve can be expressed as

$\displaystyle\max_{\boldsymbol{\alpha}}\hskip 10.0pt\mathcal{L}$ s.t.
$\displaystyle\boldsymbol{\alpha}\geq
0,\mathbf{y}^{\top}\boldsymbol{\alpha}=1$ (18)

where $\mathbf{y}$ is the vectors of labels and the Lagrangian $\mathcal{L}$
is given as Eq. 17.

###  3.5 Kernel space representation

In may practical applications, instead of a rigid boundary, a more elastic
description is favoured. In such cases, a reproducing kernel Hilbert space
representation may be adopted. Inspecting the Lagrangian in Eq. 17, it can be
observed that the training samples only appear in terms of inner products
which facilitates deriving a kernel-space representation for the proposed
approach. Since in the kernel space it holds that
$\phi(\mathbf{x}_{i})^{\top}\phi(\mathbf{x}_{j})=\kappa(\mathbf{x}_{i},\mathbf{x}_{j})$
where $\kappa(.,.)$ is the kernel function, the Lagrangian in the reproducing
kernel Hilbert space may be written as

$\displaystyle\mathcal{L}=-\bar{c_{1}}\left\lVert\boldsymbol{\alpha}\odot(1+\mathbf{y})\right\rVert_{q}^{q}-\bar{c_{2}}\left\lVert\boldsymbol{\alpha}\odot(1-\mathbf{y})\right\rVert_{q}^{q}+\sum_{i}\alpha_{i}y_{i}\kappa(\mathbf{x}_{i},\mathbf{x}_{i})-(\boldsymbol{\alpha}\odot\mathbf{y})^{\top}\mathbf{K}(\boldsymbol{\alpha}\odot\mathbf{y})$
(19)

where $\mathbf{K}$ denotes the kernel matrix. If additionally, all objects
have unit length in the feature space , i.e. if
$\phi(\mathbf{x}_{i})^{\top}\phi(\mathbf{x}_{i})=1$, one may further simplify
the Lagrangian. For this propose, note that as for normalised feature vectors
we have
$\sum_{i}\alpha_{i}y_{i}\kappa(\mathbf{x}_{i},\mathbf{x}_{i})=\mathbf{y}^{\top}\boldsymbol{\alpha}$
and since due to the constraints imposed it must hold that
$\mathbf{y}^{\top}\boldsymbol{\alpha}=1$, the term
$\sum_{i}\alpha_{i}y_{i}\kappa(\mathbf{x}_{i},\mathbf{x}_{i})$ can be safely
dropped from the objective function without affecting the result. As a result,
the optimisation problem for unit-length features shall be

$\displaystyle\min_{\boldsymbol{\alpha}}\hskip
10.0pt\bar{c_{1}}\left\lVert\boldsymbol{\alpha}\odot(1+\mathbf{y})\right\rVert_{q}^{q}$
$\displaystyle+\bar{c_{2}}\left\lVert\boldsymbol{\alpha}\odot(1-\mathbf{y})\right\rVert_{q}^{q}+(\boldsymbol{\alpha}\odot\mathbf{y})^{\top}\mathbf{K}(\boldsymbol{\alpha}\odot\mathbf{y})$
$\displaystyle\text{s.t. }\boldsymbol{\alpha}\geq
0,\mathbf{y}^{\top}\boldsymbol{\alpha}=1$ (20)

As a widely used kernel function, the Gaussian kernel by definition, yields
unit-length feature vectors in the kernel space, and the formulation above is
applicable.

###  3.6 Decision strategy

Similar to the conventional SVDD approach, for decision making in the proposed
$\ell_{p}$ slack norm method, the distance of an object to the centre of the
description is gauged and employed as a dissimilarity criterion. The distance
of an object $\mathbf{z}$ to the centre of the hypersphere $\mathcal{O}$ in
the kernel space is

$\displaystyle
f(\mathbf{z})=\left\lVert\phi(\mathbf{z})-\phi(\mathcal{O})\right\rVert_{2}^{2}=\kappa(\mathbf{z},\mathbf{z})-2\sum_{i}\alpha_{i}y_{i}\kappa(\mathbf{z},\mathbf{x}_{i})+(\boldsymbol{\alpha}\odot\mathbf{y})^{\top}\mathbf{K}(\boldsymbol{\alpha}\odot\mathbf{y})$
(21)

In order to compute the radius of the description, note that the complementary
conditions in Eqs. 14c and 14d may be compactly represented as
$\alpha_{j}(R^{2}+y_{j}\zeta_{j}-\left\lVert\phi(\mathbf{x}_{j})-\phi(\mathcal{O})\right\rVert_{2}^{2})=0$.
As a result, if for an object $\mathbf{x}_{j}$ the corresponding Lagrange
multiplier $\alpha_{j}$ is non-zero, it must hold that
$R^{2}+y_{j}\zeta_{j}-\left\lVert\phi(\mathbf{x}_{j})-\phi(\mathcal{O})\right\rVert_{2}^{2}=0$,
and hence, the radius of the description may be computed as

$\displaystyle R^{2}$
$\displaystyle=\left\lVert\phi(\mathbf{x}_{j})-\phi(\mathcal{O})\right\rVert_{2}^{2}-y_{j}\zeta_{j}$
$\displaystyle=\kappa(\mathbf{x}_{j},\mathbf{x}_{j})-2\sum_{i}\alpha_{i}y_{i}\kappa(\mathbf{x}_{j},\mathbf{x}_{i})+(\boldsymbol{\alpha}\odot\mathbf{y})^{\top}\mathbf{K}(\boldsymbol{\alpha}\odot\mathbf{y})-y_{j}\zeta_{j}$
(22)

where $j$ indexes an object whose corresponding Lagrange multiplier
$\alpha_{j}$ is non-zero. The objects whose distance to the centre of the
hyper-sphere is larger than the radius (subject to some margin) would be
classified as novel.

##  4 Generalisation error bound

In this section, using the Rademacher complexities, we characterise the
generalisation error bound for the proposed $\ell_{p}$ slack norm SVDD
approach.

###### Theorem 1

Let us assume $\mathcal{F}$ corresponds to a class of kernel-based linear
functions:

$\displaystyle\mathcal{F}=\\{x\rightarrow\mathbf{w}^{\top}\phi(x),\left\lVert\mathbf{w}\right\rVert_{2}\leq
B\\}$ (23)

then the empirical Rademacher complexity of function class $\mathcal{F}$ over
samples $(\mathbf{x}_{i})_{i=1}^{n}$, denoted as
$\hat{\mathcal{R}}_{n}(\mathcal{F})$, is bounded as [45]

$\displaystyle\hat{\mathcal{R}}_{n}(\mathcal{F})\leq\frac{2B}{n}\sqrt{tr(\mathbf{K})}\leq\frac{2BB_{\kappa}}{\sqrt{n}}$
(24)

where $tr(.)$ denotes matrix trace and $\mathbf{K}$ stands for the kernel
matrix associated with the feature mapping $\phi(.)$ and $B_{\kappa}^{2}$ is
an upper bound on the kernel function $\kappa(.,.)$.

Next, we present the main theorem concerning the generalisation performance of
the proposed approach.

###### Theorem 2

In the proposed approach, assuming that $\upsilon$ is a margin parameter, with
confidence greater than $1-\Delta$, a test point $\mathbf{x}$ is incorrectly
classified with the probability bounded as

$\displaystyle
P[y\big{(}f(\mathbf{x})-R^{2}\big{)}>\upsilon]\leq\frac{1}{n\upsilon^{p}}\left\lVert\boldsymbol{\zeta}\right\rVert_{p}^{p}+\frac{4pBB_{\kappa}}{\upsilon^{p}\sqrt{n}}(B^{2}+3B_{\kappa}^{2}+R^{2})^{p-1}+3\sqrt{\frac{\ln(2/\Delta)}{2n}}$
(25)

where $y$ is the ground truth label for observation $\mathbf{x}$.

For the proof of Theorem 2, first, we review a few relevant theories and then
present the proof.

###### Theorem 3

Assume $\delta\in(0,1)$ and suppose $\mathcal{G}$ is a function class from $X$
to $[0,1]$. Let $(\mathbf{x}_{i})_{i=1}^{n}$ be independent samples that are
drawn according to a probability distribution $\mathcal{D}$. Then with a
probability higher than $1-\Delta$ over $(\mathbf{x}_{i})_{i=1}^{n}$, for each
$g\in\mathcal{G}$ it holds that [45]

$\displaystyle\mathbb{E}_{\mathcal{D}}[g(\mathbf{x})]\leq\hat{\mathbb{E}}[g(\mathbf{x})]+\hat{\mathcal{R}}_{n}(\mathcal{G})+3\sqrt{\frac{\ln(2/\Delta)}{2n}}$
(26)

where $\hat{\mathbb{E}}[g(\mathbf{x})]$ is the empirical expectation of
$g(\mathbf{x})$ on the random sample set $(\mathbf{x}_{i})_{i=1}^{n}$ and
$\hat{\mathcal{R}}_{n}(\mathcal{G})$ denotes the empirical Rademacher
complexity of the function class $\mathcal{G}$.

###### Theorem 4

If $\mathcal{A}:\mathbb{R}\rightarrow\mathbb{R}$ is $L$-Lipschitz and
satisfies $\mathcal{A}(0)=0$, then the empirical Rademacher complexity of the
composition function class $\mathcal{A}\circ\mathcal{F}$ satisfies
$\hat{\mathcal{R}}_{n}(\mathcal{A}\circ\mathcal{F})\leq
2L\hat{\mathcal{R}}_{n}(\mathcal{F})$ [45].

Towards the proof of Theorem 2, we present the following theorem.

###### Theorem 5

Let us consider $h(\mathbf{x})$ as the hypothesis function defined as
$h(\mathbf{x})=f(\mathbf{x})-R^{2}$ where $f(\mathbf{x})$ measures the
distance of sample $\mathbf{x}$ with label $y$ to the centre of the
hypersphere in the feature space (see Eq. 21). For some fixed margin
$\upsilon>0$, we define $g(.)$ as

$\displaystyle
g(\mathbf{x})=\mathcal{A}(yh(\mathbf{x}))=\begin{cases}0&\text{if
$yh(\mathbf{x})\leq 0$;}\\\ \big{(}yh(\mathbf{x})/\upsilon\big{)}^{p}&\text{if
$0\leq yh(\mathbf{x})\leq\upsilon$;}\\\ 1&\text{else.}\end{cases}$ (27)

$\mathcal{A}:\mathbb{R}\rightarrow[0,1]$ is $L$-Lipschitz and satisfies
$\mathcal{A}(0)=0$. Then with a probability higher than $1-\Delta$ over
$(\mathbf{x}_{i})_{i=1}^{n}$ it holds

$\displaystyle\mathbb{E}_{\mathcal{D}}[g(\mathbf{x})]\leq\frac{1}{\upsilon^{p}n}\left\lVert\boldsymbol{\zeta}\right\rVert_{p}^{p}+\frac{4Bp}{n\upsilon^{p}}(B^{2}+3B_{\kappa}+R^{2})^{p-1}\sqrt{tr(\mathbf{K})}+3\sqrt{\frac{\ln(2/\Delta)}{2n}}$
(28)

Proof
We have

$\displaystyle\hat{\mathbb{E}}[g(\mathbf{x})]=\frac{1}{n}\sum_{i}g(\mathbf{x}_{i})\leq\frac{1}{n\upsilon^{p}}\sum_{i}\big{(}y_{i}(f(\mathbf{x}_{i})-R^{2})\big{)}^{p}_{+}=\frac{1}{n\upsilon^{p}}\sum_{i}\zeta_{i}^{p}=\frac{1}{n\upsilon^{p}}\left\lVert\boldsymbol{\zeta}\right\rVert_{p}^{p}$
(29)

where $(z)_{+}=\begin{cases}0&\text{if $z<0$}\\\
z&\text{otherwise.}\end{cases}$ and $\boldsymbol{\zeta}$ stands for a vector
collection of all $\zeta_{i}$’s. Note that $\mathcal{A}(.)$ is Lipschitz with
constant $L$. As with the zero-one loss, the margin loss above penalises any
misclassified objects but also penalises $h$ when it correctly classifies an
object with low confidence. In order to derive an upper bound on $L$, observe
that
$\frac{\partial\mathcal{A}}{\partial\big{(}yh(\mathbf{x})\big{)}}=\frac{p}{\upsilon^{p}}\big{(}y(f(\mathbf{x})-R^{2})\big{)}^{p-1}$,
and consequently, we have

$\displaystyle\left\lVert\frac{\partial\mathcal{A}}{\partial\big{(}yh(\mathbf{x})\big{)}}\right\rVert_{2}=\frac{p}{\upsilon^{p}}\left\lVert(f(\mathbf{x})-R^{2})\right\rVert_{2}^{p-1}\leq\frac{p}{\upsilon^{p}}(\left\lVert
f(\mathbf{x})\right\rVert_{2}+R^{2})^{p-1}$ (30)

Since the kernel function is bounded by $B_{\kappa}^{2}$, using Eq. 21, and
the fact that
$\left\lVert\mathbf{w}\right\rVert_{2}^{2}=\boldsymbol{\alpha}^{\top}\mathbf{K}\boldsymbol{\alpha}$
[45], we have

$\displaystyle\left\lVert f(\mathbf{x})\right\rVert_{2}\leq
B^{2}+3B_{\kappa}^{2}$ (31)

and hence

$\displaystyle\left\lVert\frac{\partial\mathcal{A}}{\partial\big{(}yh(\mathbf{x})\big{)}}\right\rVert_{2}\leq\frac{p}{\upsilon^{p}}(B^{2}+3B_{\kappa}^{2}+R^{2})^{p-1}$
(32)

As a result, $\mathcal{A(.)}$ is Lipschitz with constant
$L=\frac{p}{\upsilon^{p}}(B^{2}+3B_{\kappa}^{2}+R^{2})^{p-1}$.

Next, using Theorem 4 and Theorem 1, we have

$\displaystyle\hat{\mathcal{R}}_{n}(\mathcal{G})\leq
2L\hat{\mathcal{R}}_{n}(\mathcal{F})\leq\frac{4BB_{\kappa}L}{\sqrt{n}}\leq\frac{4pBB_{\kappa}}{\upsilon^{p}\sqrt{n}}(B^{2}+3B_{\kappa}^{2}+R^{2})^{p-1}$
(33)

Using Eq. 29 and Eq. 33 in Theorem 3, the proof to Theorem 5 is complete.
Since we have
$P[y\big{(}f(\mathbf{x})-R^{2}\big{)}>\upsilon]\leq\mathbb{E}_{\mathcal{D}}[g(\mathbf{x})]$,
using Theorem 5, the proof to Theorem 2 is completed. $\square$

As may be observed from Eq. 25, parameter $p$ directly affects the expected
loss on the training set (the first term on the RHS of the equation) and also
controls the Rademacher complexity (the second term on the RHS of Eq. 25) of
the proposed method. As the error probability varies as a function of $p$, the
utility of a free norm parameter in the proposed approach is justified. Note
that depending on $\boldsymbol{\zeta}$ and the margin parameter $\upsilon$,
setting $p=1$ may not minimise the RHS in Eq. 25, and hence, may lead to an
increased probability of misclassification in the proposed approach. In
practice, the norm parameter $p$ may be adjusted according to the
characteristics of the data using cross validation to optimise the performance
or to control the false acceptance/rejection rate. Note also, since parameter
$p$ appears in the dual problem as
$\left\lVert.\right\rVert^{p/(p-1)}_{p/(p-1)}$ terms (see Eq. 20), it also
affects the sparsity of $\boldsymbol{\alpha}$.

##  5 Experiments

In this section, an experimental evaluation of the proposed approach is
conducted where we provide a comparison to some other variants of the SVDD
approach as well as to baseline approaches on multiple datasets. The rest of
this section is organised as detailed next.

  * •

In Section 5.1, we visualise the decision boundaries inferred by the proposed
approach for different $p$’s for synthetic data.

  * •

In Section 5.2, the implementation details, the experimental set-up, and the
standard datasets used in the experiments are discussed.

  * •

In Section 5.3, the results of an experimental evaluation of the proposed
approach in a pure one-class setting (labelled negative objects unavailable)
are presented and compared with other methods on multiple datasets.

  * •

Section 5.4 provides the results of an experimental evaluation of the proposed
approach in the presence of negative training samples along with a comparison
against other methods on multiple datasets.

###  5.1 Decision boundaries

In order to visualise the effect of norm parameter $p$ on the inferred
decision boundaries, we randomly generate 100 normally distributed 2D samples
with a mean of 2 and standard deviation of 3 in each direction. Using a
Gaussian kernel function, the proposed approach is then run to derive a
description of the data. The experiment is repeated for different values of
$p\in\\{1,16/15,8/7,4/3,2,15/7,7/3,5/2,3\\}$ where $p=1$ corresponds to the
original SVDD method in [17]. The decision boundaries superimposed on the data
are visualised in Fig. 1. From the figure, it may be observed that for the
case of $p=1$ the method has inferred a boundary which separates a region of
relatively low density in the middle of the distribution from the rest of the
2D space. For the random data samples generated in this experiment with a mean
of $(2,2)$ this clearly indicates a case of over-fitting. By increasing the
norm parameter above 1, the decision boundary better covers the mean of the
distribution. More specifically, while for smaller values of $p$ the boundary
is tighter, for larger values the description tends to encapsulate a higher
percentage of the normal samples. As will be discussed in the following
sections, in the proposed method, we tune parameter $p$ using the validation
set corresponding to each dataset.

Figure 1: Decision boundaries for the proposed $\ell_{p}$-SVDD approach with a
Gaussian kernel function for different values of $p$ for $100$ normally
distributed random samples with mean of $2$ and standard deviation of $3$ in
each dimension. ($p=1$ corresponds to the original SVDD method).

###  5.2 Implementation details

In the experiments that follow, the features are first standardised by
subtracting the mean computed over all positive training samples and then
dividing by the standard deviation followed by normalising each feature vector
to have a unit $\ell_{2}$-norm. The positive samples are divided randomly into
three non-overlapping subsets to form the training, validation, and the test
sets. Similarly, the negative samples are divided randomly into three disjoint
subsets for training, validation and testing purposes. In order to minimise
possible effects of random data partitioning on the performance, we repeat the
procedure above 10 times, and record the mean along with the standard
deviation of the performance over these 10 trials. We set the parameters of
all methods on the corresponding validation subset of each dataset. In
particular, for the proposed approach
$p\in\\{32/31,16/15,8/7,6/5,4/3,3/2,2,5/2,5,20\\}$ and
$c_{1},c_{2}\in\\{10^{-3},10^{-2},10^{-1},1\\}$. In all experiments, we use a
Gaussian kernel the width of which is set to half of the average pairwise
Euclidean distance among all training objects. As the dual problem in Eq. 20
is convex, one may use different algorithms for optimisation. In this work, we
use CVX [46], a package for solving convex programmes.

Table 1: Statistics of the datasets used. Abbrev. | Dataset | #total | #positive | dim. | Source
---|---|---|---|---|---
D1 | Iris (virginica) | 150 | 50 | 4 | UCI
D2 | Hepatitis (normal) | 155 | 123 | 19 | UCI
D3 | Ecoli (periplasm) | 336 | 52 | 7 | UCI
D4 | Concordia16 (digit 1) | 4000 | 400 | 256 | CENPARMI
D5 | Delft pump (5x1) | 336 | 133 | 64 | Delft
D6 | Balance-scale (middle) | 625 | 49 | 4 | UCI
D7 | Wine (2) | 178 | 71 | 13 | UCI
D8 | Waveform (0) | 900 | 300 | 21 | UCI
D9 |  Survival ($<$5yr) | 306 | 81 | 3 | UCI
D10 |  Housing (MEDV$>$35) | 506 | 48 | 13 | Statlib
D11 | glass6 | 214 | 185 | 9 | KEEL
D12 | haberman | 306 | 225 | 3 | KEEL
D13 | led7digit | 443 | 406 | 7 | KEEL
D14 | pima | 768 | 500 | 8 | KEEL
D15 | wisconsin | 683 | 444 | 9 | KEEL
D16 | yeast(0-5-6-7-9-vs-4) | 528 | 477 | 8 | KEEL
D17 | cleveland(0-vs-4) | 177 | 164 | 13 | KEEL
D18 | Breast(benign) | 699 | 458 | 9 | UCI
D19 |  Survival ($>$5yr) | 306 | 225 | 3 | UCI
D20 | FMNIST (Class ”1”) | 1926 | 1027 | 784 | Zalando

In order to evaluate the proposed approach, 20 benchmark databases from the
UCI repository [47], TUDelf University [48], KEEL repository [49], CENPARMI
[50], Statlib [51], and Zalando [52] are used. The datasets used in the
experiments correspond to different application domains from varied sources.
The statistics of the datasets used in the experiments are reported in Table
LABEL:dstats. For the evaluation of the proposed approach, we conduct two sets
of experiments. The first set follows a pure one-class classification
paradigm, i.e. only positive samples are used to train the models. In the
second set of experiments, negative objects are also deployed for model
training. For comparison, we report the performance of the original SVDD
approach of [17] denoted as ”$\ell_{1}$-SVDD” and also its alternative variant
which considers squared errors in the objective function, denoted as
”$\ell_{2}$-SVDD” [44]. The proposed approach is denoted as ”$\ell_{p}$-SVDD”
in the corresponding tables. We also provide a comparison of the proposed
$\ell_{p}$-SVDD method to some linear re-weighting variants of the SVDD
approach including P-SVDD [36], DW-SVDD [37], and GL-SVDD [35] as well as
state-of-the-art OCC techniques. In particular, we have included kernel-based
one-class classifiers which are applicable to moderately-sized datasets. These
are the kernel Gaussian Process method (GP) [33], the Kernel Null Foley-Sammon
Transform (KNFST) [53, 32], and the Kernel Principal Component Analysis (KPCA)
[28].

Following the common approach in the literature and in order to facilitate the
comparison of the performances of different methods independent from a
specific operating threshold, we report the performances in terms of the AUC
measure which is the area under the Receiver Operating Characteristic curves
(ROC). The ROC curve characterises the true positive rate against the false
positive rate at various operating thresholds. A higher AUC indicates a better
performance for the system.

Table 2: Comparison of the performance of OCC approaches on different datasets in a pure one-class scenario in terms of $\%$AUC. | GP | KPCA | KNFST | P-SVDD | DW-SVDD | GL-SVDD |  $\ell_{1}$-SVDD |  $\ell_{2}$-SVDD |  $\ell_{p}$-SVDD
---|---|---|---|---|---|---|---|---|---
D1 |  68.03$\pm$16.51 |  78.17$\pm$22.69 |  68.51$\pm$16.46 |  78.98$\pm$15.58 |  67.42$\pm$16.75 |  67.70$\pm$16.39 |  67.42$\pm$16.75 |  67.37$\pm$16.52 | $\mathbf{81.23\pm 11.26}$
D2 |  58.78$\pm$4.37 |  66.46$\pm$6.75 |  58.65$\pm$4.39 |  62.51$\pm$5.98 |  59.36$\pm$4.37 |  61.20$\pm$6.06 |  59.98$\pm$4.31 |  59.48$\pm$4.26 | $\mathbf{66.60\pm 6.80}$
D3 |  61.30$\pm$11.33 |  53.06$\pm$13.66 |  61.32$\pm$11.15 |  53.98$\pm$12.31 |  61.67$\pm$11.66 |  61.08$\pm$10.49 |  61.08$\pm$11.36 |  61.20$\pm$11.46 | $\mathbf{62.19\pm 10.53}$
D4 |  92.96$\pm$2.60 |  94.10$\pm$1.73 |  92.94$\pm$2.62 |  93.24$\pm$2.50 |  93.31$\pm$2.54 |  93.21$\pm$2.53 |  94.36$\pm$1.84 |  93.58$\pm$2.42 | $\mathbf{94.43\pm 1.58}$
D5 |  87.87$\pm$3.43 |  86.07$\pm$3.29 |  87.85$\pm$3.45 |  84.78$\pm$3.92 |  87.85$\pm$3.45 |  88.02$\pm$3.33 |  87.85$\pm$3.45 |  87.90$\pm$3.40 | $\mathbf{88.41\pm 2.77}$
D6 |  94.52$\pm$3.57 |  90.77$\pm$2.46 |  94.66$\pm$3.52 |  92.94$\pm$3.16 |  94.68$\pm$3.54 |  94.64$\pm$3.54 |  94.70$\pm$3.55 |  94.60$\pm$3.56 | $\mathbf{94.79\pm 3.59}$
D7 |  60.82$\pm$11.03 |  72.30$\pm$11.10 |  60.71$\pm$10.99 |  65.64$\pm$9.04 |  61.12$\pm$11.57 |  64.12$\pm$10.85 |  60.79$\pm$11.04 |  60.90$\pm$11.19 | $\mathbf{72.53\pm 11.36}$
D8 |  64.15$\pm$4.66 |  61.47$\pm$1.90 |  64.10$\pm$4.64 |  64.09$\pm$5.91 |  65.37$\pm$5.09 |  65.39$\pm$5.41 |  64.91$\pm$5.53 |  65.10$\pm$5.48 | $\mathbf{65.86\pm 4.23}$
D9 |  53.41$\pm$18.99 |  47.37$\pm$14.10 |  59.66$\pm$16.34 |  52.29$\pm$19.26 |  58.28$\pm$21.15 |  66.40$\pm$12.11 |  64.43$\pm$14.87 |  60.72$\pm$15.26 | $\mathbf{67.40\pm 10.21}$
D10 |  87.73$\pm$6.00 |  78.70$\pm$8.49 |  87.76$\pm$6.02 |  81.28$\pm$11.89 |  87.77$\pm$6.11 |  87.82$\pm$6.03 |  87.77$\pm$6.11 |  87.78$\pm$6.08 | $\mathbf{87.91\pm 5.92}$
D11 |  87.81$\pm$10.11 |  95.44$\pm$2.59 |  87.55$\pm$10.65 |  95.48$\pm$3.53 |  86.90$\pm$10.06 |  96.58$\pm$2.18 |  94.24$\pm$2.53 |  90.68$\pm$5.67 | $\mathbf{96.73\pm 1.38}$
D12 |  59.70$\pm$5.90 |  70.66$\pm$5.79 |  55.54$\pm$5.51 |  59.75$\pm$7.79 |  55.84$\pm$7.12 | $\mathbf{70.77\pm 5.96}$ |  58.34$\pm$7.41 |  66.02$\pm$4.94 | $\mathbf{70.77\pm 5.88}$
D13 |  62.63$\pm$18.55 |  67.49$\pm$7.79 |  62.34$\pm$18.01 |  66.60$\pm$15.73 |  41.54$\pm$6.43 |  66.77$\pm$14.25 |  62.84$\pm$13.62 |  69.04$\pm$10.99 | $\mathbf{69.41\pm 9.62}$
D14 |  51.66$\pm$5.36 | $\mathbf{71.43\pm 3.41}$ |  51.46$\pm$5.32 |  53.47$\pm$5.42 |  54.93$\pm$4.78 |  56.04$\pm$4.46 |  61.24$\pm$4.95 |  59.83$\pm$3.75 | $\mathbf{71.43\pm 3.57}$
D15 |  52.70$\pm$12.37 |  95.88$\pm$1.05 |  53.51$\pm$12.56 |  65.21$\pm$12.28 |  47.10$\pm$14.97 |  70.68$\pm$10.68 |  93.39$\pm$1.73 |  68.75$\pm$10.83 | $\mathbf{95.91\pm 1.03}$
D16 |  57.74$\pm$9.93 |  63.71$\pm$7.39 |  58.56$\pm$8.80 |  62.94$\pm$8.48 |  61.02$\pm$10.33 |  63.26$\pm$9.98 |  60.93$\pm$9.65 |  61.52$\pm$10.68 | $\mathbf{64.23\pm 8.29}$
D17 |  51.49$\pm$13.99 |  79.13$\pm$9.71 |  51.78$\pm$14.04 |  57.93$\pm$13.99 |  51.09$\pm$13.90 |  51.85$\pm$14.47 |  51.09$\pm$13.90 |  51.35$\pm$14.06 | $\mathbf{79.38\pm 9.27}$
D18 | $\mathbf{45.02\pm 3.25}$ |  38.11$\pm$2.53 |  45.74$\pm$2.93 |  41.15$\pm$5.13 |  42.64$\pm$5.04 |  42.07$\pm$5.14 |  41.22$\pm$5.62 |  41.99$\pm$5.16 |  41.76$\pm$5.67
D19 |  55.16$\pm$13.36 |  37.01$\pm$8.20 |  44.70$\pm$11.81 |  51.49$\pm$13.02 |  61.38$\pm$14.80 |  61.01$\pm$9.08 |  60.28$\pm$9.47 |  58.64$\pm$11.56 | $\mathbf{62.60\pm 9.83}$
D20 |  94.82$\pm$1.47 |  94.86$\pm$10.9 |  94.78$\pm$1.49 |  94.96$\pm$1.23 |  95.37$\pm$0.87 | $\mathbf{96.92\pm 0.73}$ |  96.37$\pm$0.79 |  96.48$\pm$1.14 | $\mathbf{96.92\pm 0.63}$

###  5.3 Pure one-class setting

In this setting, only positive objects are used for training. Table 2 reports
the performances of different methods in this setting where we set parameter
$p$ on the validation subset of each dataset to maximise the performance. A
number of observations from the table are in order. First, on all datasets the
proposed $\ell_{p}$-SVDD approach yields a superior performance compared to
its $\ell_{1}$-SVDD and $\ell_{2}$-SVDD variants. In particular, on some
datasets such as D1 and D14, the improvement in the performance offered by the
proposed approach is substantial while on some other datasets such as D17 the
improvement is huge and reaches $28\%$. It should be noted that the
performance improvements offered by the proposed approach are obtained despite
the fact that the validation sets of some datasets may not be very large, and
hence, may not serve as a very good representative of the entire the
distribution of samples for tuning parameter $p$. It is expected that a more
representative validation set would lead to even further improvements in the
performance. A statistical ranking of different methods in the pure one-class
setting using the Friedman’s test is provided in Table 3. From the table, it
can be observed that while the proposed $\ell_{p}$-SVDD approach ranks the
best among other approaches while the standard $\ell_{1}$-SVDD approach ranks
much worst which underlines the significance of the proposed $\ell_{p}$ slack
norm approach. Furthermore, although the $\ell_{2}$-SVDD method provides some
improvement with respect to the original $\ell_{1}$-SVDD approach, its
performance is still inferior compared to the proposed method. The second best
performing method (in terms of average ranking) corresponds to a sample re-
weighting SVDD approach presented in [35] which uses global and local
statistics to linearly weight slacks in the objective function.

Table 3: Average ranking of different OCC methods in a pure one-class setting using the Friedman’s test. (p-value=$5.65e-10$) Algorithm | Rank
---|---
GP | 6.50
KPCA | 5.30
KNFST | 6.70
P-SVDD | 5.85
DW-SVDD | 5.72
GL-SVDD | 3.65
$\ell_{1}$-SVDD | 5.20
$\ell_{2}$-SVDD | 4.80
$\ell_{p}$-SVDD (this work) | $\mathbf{1.27}$

###  5.4 Training in the presence of negative data

In this second evaluation setting, in addition to positive objects, labelled
negative samples are also used for training. Table 4 reports the performances
of different methods in this setting. Note that as in the case of pure one-
class learning, the optimal $p$ value for the proposed approach is determined
on the validation set. From among the GP, KPCA and KNFST approaches, only the
KNFST approach is able to directly deploy negative samples for training. In
order to emphasise that a method uses negative objects for training, a
negative exponent (”-”) is used in the table. We also include the P-SVDD, DW-
SVDD, and the GL-SVDD approaches trained using both negative and positive
samples and denote them as P-SVDD-, DW-SVDD-, and GL-SVDD-. From Table 4, it
can be observed that on all datasets the proposed $\ell_{p}$-SVDD approach
obtains a better performance as compared with its $\ell_{1}$-SVDD and
$\ell_{2}$-SVDD variants. In particular, while on some datasets the
$\ell_{1}$-SVDD and $\ell_{2}$-SVDD approaches are unable to effectively
utilise negative training samples, the proposed $\ell_{p}$-SVDD method can
better benefit from such samples to refine the description for improved
performance. When compared with linear sample re-weighting methods of P-SVDD-,
DW-SVDD-, and GL-SVDD-, the proposed approach also performs better. An average
ranking of different methods in this evaluation setting is provided in Table
5. From Table 5 it may be seen that the proposed $\ell_{p}$-SVDD- approach
utilising negative objects for training ranks the best among other
competitors. Furthermore, neither the $\ell_{1}$-SVDD- nor the
$\ell_{2}$-SVDD- methods which use negative training samples do not rank the
second. The second best performing method in this setting corresponds to the
KNFST method [53, 32].

Table 4: Comparison of the performance of OCC approaches on different datasets in the presence of negative training objects in terms of $\%$AUC. Dataset |  KNFST- |  P-SVDD- |  DW-SVDD- |  GL-SVDD- |  $\ell_{1}$-SVDD- |  $\ell_{2}$-SVDD- |  $\ell_{p}$-SVDD-
---|---|---|---|---|---|---|---
D1 |  95.90$\pm$5.48 |  39.45$\pm$21.77 |  74.38$\pm$24.80 |  64.62$\pm$19.09 |  58.53$\pm$21.63 |  59.79$\pm$22.50 | $\mathbf{100.00\pm 0.00}$
D2 |  69.25$\pm$11.91 |  55.88$\pm$4.63 |  58.98$\pm$5.02 |  66.41$\pm$6.91 |  59.38$\pm$4.99 |  59.60$\pm$5.12 | $\mathbf{71.57\pm 7.92}$
D3 |  70.80$\pm$4.94 |  52.53$\pm$8.94 |  60.70$\pm$7.55 |  60.71$\pm$8.34 |  59.19$\pm$8.86 |  59.95$\pm$8.56 | $\mathbf{75.82\pm 5.37}$
D4 | $\mathbf{96.56\pm 1.50}$ |  93.27$\pm$1.71 |  94.92$\pm$1.45 |  93.33$\pm$1.79 |  94.46$\pm$1.23 |  93.88$\pm$1.23 | $\mathbf{96.56\pm 1.43}$
D4 |  93.02$\pm$2.11 |  88.63$\pm$5.07 |  91.17$\pm$3.49 |  91.17$\pm$3.49 |  91.17$\pm$3.49 |  91.19$\pm$3.45 | $\mathbf{93.32\pm 1.94}$
D6 |  90.29$\pm$12.84 |  88.73$\pm$3.46 |  92.61$\pm$4.15 |  92.56$\pm$4.14 |  92.62$\pm$4.14 |  92.41$\pm$4.23 | $\mathbf{96.11\pm 4.79}$
D7 |  93.70$\pm$3.77 |  50.84$\pm$8.34 |  73.84$\pm$6.50 |  58.59$\pm$8.39 |  58.13$\pm$9.14 |  58.66$\pm$8.90 | $\mathbf{94.83\pm 3.00}$
D8 |  90.01$\pm$1.31 |  64.97$\pm$3.74 |  66.93$\pm$3.24 |  67.20$\pm$3.15 |  66.95$\pm$3.30 |  65.95$\pm$3.35 | $\mathbf{91.51\pm 1.40}$
D9 |  64.14$\pm$9.19 |  48.56$\pm$10.65 |  83.08$\pm$10.33 |  63.30$\pm$11.58 |  59.92$\pm$9.96 |  60.72$\pm$15.15 | $\mathbf{96.44\pm 2.25}$
D10 |  89.37$\pm$7.15 |  82.92$\pm$7.86 |  86.66$\pm$8.53 |  85.88$\pm$8.47 |  86.66$\pm$8.53 |  86.58$\pm$8.58 | $\mathbf{89.81\pm 5.00}$
D11 |  96.52$\pm$3.62 |  84.95$\pm$11.15 |  94.90$\pm$1.78 |  96.50$\pm$1.86 |  94.02$\pm$3.96 |  89.88$\pm$5.76 | $\mathbf{97.12\pm 1.72}$
D12 |  52.52$\pm$7.84 |  58.84$\pm$6.19 |  63.06$\pm$5.22 |  69.44$\pm$7.54 |  62.10$\pm$9.37 |  65.95$\pm$6.16 | $\mathbf{71.23\pm 7.42}$
D13 |  91.11$\pm$7.62 |  68.97$\pm$12.57 |  47.65$\pm$13.31 |  69.24$\pm$13.25 |  69.54$\pm$10.59 |  66.53$\pm$8.41 | $\mathbf{95.20\pm 2.93}$
D14 |  63.94$\pm$3.85 |  55.37$\pm$4.12 |  64.88$\pm$4.01 |  61.15$\pm$4.01 |  59.14$\pm$4.89 |  64.90$\pm$3.59 | $\mathbf{79.75\pm 0.99}$
D15 |  94.20$\pm$2.85 |  68.65$\pm$4.92 |  65.80$\pm$8.32 |  91.91$\pm$2.71 |  93.32$\pm$2.35 |  88.39$\pm$3.73 | $\mathbf{98.69\pm 0.54}$
D16 |  61.74$\pm$12.47 |  64.12$\pm$5.56 |  63.19$\pm$6.27 |  64.67$\pm$5.55 |  64.33$\pm$5.08 |  62.68$\pm$7.29 | $\mathbf{84.70\pm 2.69}$
D17 |  88.11$\pm$6.10 |  51.27$\pm$14.69 |  50.22$\pm$13.93 |  51.75$\pm$14.62 |  50.22$\pm$13.93 |  50.36$\pm$13.67 | $\mathbf{92.80\pm 3.73}$
D18 |  56.83$\pm$3.88 |  49.04$\pm$1.92 |  51.52$\pm$2.54 |  51.74$\pm$3.55 |  52.02$\pm$2.40 |  47.06$\pm$4.54 | $\mathbf{66.10\pm 9.22}$
D19 |  78.46$\pm$4.54 |  62.56$\pm$9.53 |  0.8586$\pm$10.37 |  65.30$\pm$9.56 |  66.69$\pm$9.38 |  50.44$\pm$12.80 | $\mathbf{92.65\pm 2.13}$
D20 | $\mathbf{99.25\pm 0.40}$ |  95.83$\pm$1.40 |  97.72$\pm$0.99 |  97.55$\pm$0.79 |  97.03$\pm$1.15 |  97.24$\pm$0.96 |  98.07$\pm$0.86
Table 5: Average ranking of different OCC methods in the presence of negative training samples using the Friedman’s test). (p-value=$8.97e-14$) Algorithm | Rank
---|---
KNFST- | 2.80
P-SVDD- | 6.350
DW-SVDD- | 4.45
GL-SVDD- | 3.90
$\ell_{1}$-SVDD- | 4.55
$\ell_{2}$-SVDD- | 4.90
$\ell_{p}$-SVDD- (this work) | $\mathbf{1.05}$

###  5.5 Discussion

From a pattern classification perspective, our observations that an
$\ell_{1}$-norm error function may not be the best choice for one-class
classification are consistent with the findings in two/multi-class
classification setting [18] where it is verified that error encoding
mechanisms other than the widely used $\ell_{1}$-norm may yield better
generalisation capabilities for the standard two-class SVM. Note that, while
in [18] only the specific cases of $p=2$ and $p=\infty$ are analysed for the
two/multi-class scenario, in this work, we formulated a generic
$\ell_{p}$-norm error function for the one-class setting, and as demonstrated
in the experimental evaluation section, the proposed approach yields better
classification performance compared to the conventional $\ell_{1}$, $\ell_{2}$
and also sample re-weighting approaches for SVDD.

It should be noted that although the proposed approach has been targeted
towards learning data descriptions on medium-sized datasets, yet, it may be
extended to work in conjunction with deep learning methods on large-scale
datasets to benefit from the availability of such data. In this context, one
possible direction for the extension of the proposed approach is to couple the
training stage of the proposed $\ell_{p}$-SVDD approach with that of a deep
network, akin to the DeepSVDD technique [11], for a joint learning of data
representation and one-class description learning for enhanced performance.

While in the proposed approach all slacks are similarly penalised via a common
$p$-norm cost, another possibility is to consider sample-specific non-linear
penalty functions, for instance by further weighting each object according to
some criteria, similar in spirit to the work in [36, 37, 38, 39]. Finally,
there exists multi-class classification problems where a test observation may
potentially belong to none of the predefined classes, giving rise to an open-
set classification problem [54]. In this context, one may consider an
extension of the proposed approach where multiple hyper-spheres, each
corresponding to a single class are inferred.

##  6 Conclusion

We generalised the SVDD approach from an $\ell_{1}$-norm risk to an
$\ell_{p}$-norm ($p\geq 1$) cost w.r.t. slacks, enabling formulating non-
linear cost functions over errors. An $\ell_{p}$-norm ($p\geq 1$) penalty
function is a direct generalisation of the $\ell_{1}$ penalty term, and thus,
the proposed approach presented a natural and intuitive generalisation of the
SVDD method. By virtue of a dual representation of the problem, it was shown
that the proposed approach corresponds to an optimisation task where sparsity-
inducing $\ell_{q}$ norms ($q=p/(p-1)$) over the dual variable (i.e.
$\boldsymbol{\alpha}$) (absent in the original SVDD formulation) are
introduced into the objective function, allowing the algorithm to tune into
the inherent sparsity of the data. A theoretical analysis of the proposed
approach based on Rademacher complexities yielded an upper bound on the
classification error, emphasising the dependence of the performance on the
norm parameter $p$. The experimental evaluation on several standard OCC
datasets, on the other hand, revealed that the proposed approach leads to
improvements upon the existing $\ell_{1}$ and $\ell_{2}$ penalty functions in
addition to some other linear sample re-weighting SVDD variants and also
performs very favourably compared with other existing OCC methods both in a
pure one-class setting and also in the case of training in the presence of
labelled negative objects. Finally, we discussed several future research
avenues regarding possible extensions of the proposed approach.

## Acknowledgements

This research is supported by The Scientific and Technological Research
Council of Turkey (TÜBİTAK) under the grant no 121E465.

## References

  * [1] S. Fatemifar, S. R. Arashloo, M. Awais, J. Kittler, Client-specific anomaly detection for face presentation attack detection, Pattern Recognition 112 (2021) 107696.
  * [2] D. Carrera, B. Rossi, P. Fragneto, G. Boracchi, Online anomaly detection for long-term ecg monitoring using wearable devices, Pattern Recognition 88 (2019) 482–492.
  * [3] A. Rabaoui, M. Davy, S. Rossignol, N. Ellouze, Using one-class svms and wavelets for audio surveillance, IEEE Transactions on Information Forensics and Security 3 (4) (2008) 763–775.  doi:10.1109/TIFS.2008.2008216.
  * [4] X. Zhang, S. Yang, J. Zhang, W. Zhang, Video anomaly detection and localization using motion-field shape description and homogeneity testing, Pattern Recognition 105 (2020) 107394.
  * [5] P. Nader, P. Honeine, P. Beauseroy, lp-norms in one-class classification for intrusion detection in scada systems, IEEE Transactions on Industrial Informatics 10 (4) (2014) 2308–2317.  doi:10.1109/TII.2014.2330796.
  * [6] R. Chaker, Z. A. Aghbari, I. N. Junejo, Social network model for crowd anomaly detection and localization, Pattern Recognition 61 (2017) 266–281.
  * [7] S. Budalakoti, A. N. Srivastava, M. E. Otey, Anomaly detection and diagnosis algorithms for discrete symbol sequences with applications to airline safety, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 39 (1) (2009) 101–113.  doi:10.1109/TSMCC.2008.2007248.
  * [8] S. Kamaruddin, V. Ravi, Credit card fraud detection using big data analytics: Use of psoaann based one-class classification, in: Proceedings of the International Conference on Informatics and Analytics, ICIA-16, ACM, New York, NY, USA, 2016, pp. 33:1–33:8.  doi:10.1145/2980258.2980319.
  * [9] G. G. Sundarkumar, V. Ravi, V. Siddeshwar, One-class support vector machine based undersampling: Application to churn prediction and insurance fraud detection, in: 2015 IEEE International Conference on Computational Intelligence and Computing Research (ICCIC), 2015, pp. 1–7.  doi:10.1109/ICCIC.2015.7435726.
  * [10] I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, MIT Press, 2016, http://www.deeplearningbook.org.
  * [11] L. Ruff, R. Vandermeulen, N. Goernitz, L. Deecke, S. A. Siddiqui, A. Binder, E. Müller, M. Kloft, Deep one-class classification, in: J. Dy, A. Krause (Eds.), Proceedings of the 35th International Conference on Machine Learning, Vol. 80 of Proceedings of Machine Learning Research, PMLR, 2018, pp. 4393–4402.
URL https://proceedings.mlr.press/v80/ruff18a.html

  * [12] S. M. Erfani, S. Rajasegarar, S. Karunasekera, C. Leckie, High-dimensional and large-scale anomaly detection using a linear one-class svm with deep learning, Pattern Recognition 58 (2016) 121–134.
  * [13] B. Settles, Active Learning, Synthesis Lectures on Artificial Intelligence and Machine Learning, Morgan and Claypool Publishers, 2012.
  * [14] V. Vapnik, R. Izmailov, Learning using privileged information: Similarity control and knowledge transfer, Journal of Machine Learning Research 16 (61) (2015) 2023–2049.
URL http://jmlr.org/papers/v16/vapnik15b.html

  * [15] C. Cortes, V. Vapnik, Support-vector networks, Machine Learning 20 (3) (1995) 273–297.  doi:10.1007/BF00994018.
  * [16] V. N. Vapnik, Statistical Learning Theory, Wiley-Interscience, 1998.
  * [17] D. M. Tax, R. P. Duin, Support vector data description, Machine Learning 54 (1) (2004) 45–66.  doi:10.1023/B:MACH.0000008084.60811.49.
  * [18] V. Vapnik, R. Izmailov, Reinforced svm method and memorization mechanisms, Pattern Recognition 119 (2021) 108018.
  * [19] V. Chandola, A. Banerjee, V. Kumar, Anomaly detection: A survey, ACM Comput. Surv. 41 (3).
  * [20] S. S. Khan, M. G. Madden, One-class classification: taxonomy of study and review of techniques, The Knowledge Engineering Review 29 (3) (2014) 345–374.  doi:10.1017/S026988891300043X.
  * [21] M. A. Pimentel, D. A. Clifton, L. Clifton, L. Tarassenko, A review of novelty detection, Signal Processing 99 (2014) 215–249.
  * [22] P. Perera, P. Oza, V. M. Patel, One-class classification: A survey, CoRR abs/2101.03064.  arXiv:2101.03064.
URL https://arxiv.org/abs/2101.03064

  * [23] J. Kittler, W. Christmas, T. de Campos, D. Windridge, F. Yan, J. Illingworth, M. Osman, Domain anomaly detection in machine perception: A system architecture and taxonomy, IEEE Transactions on Pattern Analysis and Machine Intelligence 36 (5) (2014) 845–859.  doi:10.1109/TPAMI.2013.209.
  * [24] C. Bishop, Novelty detection and neural network validation, IEE Proceedings - Vision, Image and Signal Processing 141 (1994) 217–222(5).
  * [25] L. Parra, G. Deco, S. Miesbach, Statistical independence and novelty detection with information preserving nonlinear maps, Neural Computation 8 (2) (1996) 260–269.  doi:10.1162/neco.1996.8.2.260.
  * [26] E.-S. Platzer, J. Denzler, H. SuBe, J. Nagele, K.-H. Wehking, Challenging anomaly detection in wire ropes using linear prediction combined with one-class classification., in: O. Deussen, D. A. Keim, D. Saupe (Eds.), VMV, Aka GmbH, 2008, pp. 343–352.
  * [27] S. Fatemifar, M. Awais, A. Akbari, J. Kittler, Developing a generic framework for anomaly detection, Pattern Recognition 124 (2022) 108500.
  * [28] H. Hoffmann, Kernel pca for novelty detection, Pattern Recognition 40 (3) (2007) 863 – 874.
  * [29] Y. Xiao, H. Wang, W. Xu, J. Zhou, L1 norm based kpca for novelty detection, Pattern Recognition 46 (1) (2013) 389–396.
  * [30] D. Abati, A. Porrello, S. Calderara, R. Cucchiara, Latent space autoregression for novelty detection, in: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 481–490.  doi:10.1109/CVPR.2019.00057.
  * [31] P. Bodesheim, A. Freytag, E. Rodner, M. Kemmler, J. Denzler, Kernel null space methods for novelty detection, in: 2013 IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 3374–3381.  doi:10.1109/CVPR.2013.433.
  * [32] S. R. Arashloo, J. Kittler, Robust one-class kernel spectral regression, IEEE Transactions on Neural Networks and Learning Systems 32 (3) (2021) 999–1013.  doi:10.1109/TNNLS.2020.2979823.
  * [33] M. Kemmler, E. Rodner, E.-S. Wacker, J. Denzler, One-class classification with gaussian processes, Pattern Recognition 46 (12) (2013) 3507–3518.
  * [34] D. Tax, R. Duin, Data description in subspaces, in: Proceedings 15th International Conference on Pattern Recognition. ICPR-2000, Vol. 2, 2000, pp. 672–675 vol.2.  doi:10.1109/ICPR.2000.906164.
  * [35] W. Hu, T. Hu, Y. Wei, J. Lou, S. Wang, Global plus local jointly regularized support vector data description for novelty detection, IEEE Transactions on Neural Networks and Learning Systems (2021) 1–13doi:10.1109/TNNLS.2021.3129321.
  * [36] C.-D. Wang, J. Lai, Position regularized support vector domain description, Pattern Recognition 46 (3) (2013) 875–884.
  * [37] M. Cha, J. S. Kim, J.-G. Baek, Density weighted support vector data description, Expert Systems with Applications 41 (7) (2014) 3343–3350.
  * [38] K. Lee, D.-W. Kim, D. Lee, K. H. Lee, Improving support vector data description using local density degree, Pattern Recognition 38 (10) (2005) 1768–1771.
  * [39] G. Chen, X. Zhang, Z. J. Wang, F. Li, Robust support vector data description for outlier detection with noise or uncertain data, Knowledge-Based Systems 90 (2015) 129–137.
  * [40] M. Wu, J. Ye, A small sphere and large margin approach for novelty detection using training data with outliers, IEEE Transactions on Pattern Analysis and Machine Intelligence 31 (11) (2009) 2088–2092.  doi:10.1109/TPAMI.2009.24.
  * [41] G. Huang, H. Chen, Z. Zhou, F. Yin, K. Guo, Two-class support vector data description, Pattern Recognition 44 (2) (2011) 320–329.
  * [42] M. Turkoz, S. Kim, Y. Son, M. K. Jeong, E. A. Elsayed, Generalized support vector data description for anomaly detection, Pattern Recognition 100 (2020) 107119\.
  * [43] I. W. Tsang, J. T. Kwok, P.-M. Cheung, Core vector machines: Fast svm training on very large data sets, Journal of Machine Learning Research 6 (13) (2005) 363–392.
URL http://jmlr.org/papers/v6/tsang05a.html

  * [44] W. cheng Chang, C. pei Lee, C. jen Lin, A revisit to support vector data description (svdd), Implemented in the LIBSVM extension for SVDD (2015).
  * [45] J. Shawe-Taylor, N. Cristianini, Kernel Methods for Pattern Analysis, Cambridge University Press, 2004.  doi:10.1017/CBO9780511809682.
  * [46] M. Grant, S. Boyd, CVX: Matlab software for disciplined convex programming, version 2.1, http://cvxr.com/cvx (Mar. 2014).
  * [47] D. Dua, C. Graff, UCI machine learning repository (2017).
URL http://archive.ics.uci.edu/ml

  * [48] D. M. J. Tax, R. P. W. Duin, Characterizing one-class datasets, 2006.
  * [49] J. Alcala-Fdez, A. Fernandez, J. Luengo, J. Derrac, S. Garcia, Keel data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework., J. Multiple Valued Log. Soft Comput. 17 (2-3) (2011) 255–287.
  * [50] S.-B. Cho, Neural-network classifiers for recognizing totally unconstrained handwritten numerals, Trans. Neur. Netw. 8 (1) (1997) 43–53.
  * [51] D. Harrison, D. L. Rubinfeld, Hedonic housing prices and the demand for clean air, Journal of Environmental Economics and Management 5 (1) (1978) 81–102.  doi:https://doi.org/10.1016/0095-0696(78)90006-2.
  * [52] H. Xiao, K. Rasul, R. Vollgraf, Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms (2017).  arXiv:cs.LG/1708.07747.
  * [53] Y. Lin, G. Gu, H. Liu, J. Shen, Kernel null foley-sammon transform, in: 2008 International Conference on Computer Science and Software Engineering, Vol. 1, 2008, pp. 981–984.  doi:10.1109/CSSE.2008.621.
  * [54] W. J. Scheirer, A. de Rezende Rocha, A. Sapkota, T. E. Boult, Toward open set recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence 35 (7) (2013) 1757–1772.  doi:10.1109/TPAMI.2012.256.