# Unknown Face Presentation Attack Detection via Localised Learning of
Multiple Kernels

Shervin Rahimzadeh Arashloo S.R. Arashloo is with the centre for vision,
speech, and signal processing (CVSSP), University of Surrey, Guildford,
Surrey, UK. e-mail<EMAIL_ADDRESS>received ? ?, 2022;
revised ? ?, 2022.

###### Abstract

The paper studies face spoofing, a.k.a. presentation attack detection (PAD) in
the demanding scenarios of unknown types of attack. While earlier studies have
revealed the benefits of ensemble methods, and in particular, a multiple
kernel learning approach to the problem, one limitation of such techniques is
that they typically treat the entire observation space similarly and ignore
any variability and local structure inherent to the data. This work studies
this aspect of the face presentation attack detection problem in relation to
multiple kernel learning in a one-class setting to benefit from intrinsic
local structure in bona fide face samples. More concretely, inspired by the
success of the one-class Fisher null formalism, we formulate a convex
localised multiple kernel learning algorithm by imposing a joint matrix-norm
constraint on the collection of local kernel weights and infer locally
adaptive weights for zero-shot one-class unseen attack detection.

We present a theoretical study of the proposed localised MKL algorithm using
Rademacher complexities to characterise its generalisation capability and
demonstrate the advantages of the proposed technique over some other options.
An assessment of the proposed approach on general object image datasets
illustrates its efficacy for abnormality and novelty detection while the
results of the experiments on face PAD datasets verifies its potential in
detecting unknown/unseen face presentation attacks.

###### Index Terms:

unknown face presentation attack/spoofing detection, one-class multiple kernel
learning, localised learning, generalisation analysis.

††publicationid: pubid:

##  I Introduction

When exposed to sensor-level attacks, known as presentation/spoofing attacks,
the trustworthiness of face recognition systems [1, 2, 3, 4] is significantly
questioned, necessitating the development of robust presentation attack
detection algorithms. Despite big advances, the current face PAD approaches
have their own limitations. One source of difficulty in detecting presentation
attacks is the diversity in attack samples which may be attributed to using a
variety of different presentation attack mechanisms such as print, replay,
mask, make-up, or to other environmental imaging conditions resulting in a
significant within-class variance that may erode the separability between
classes. Unseen/unknown attacks that were not accessible during the system’s
training phase represent another, arguably a severe challenge in face PAD,
rendering it as an open-set classification problem [5]. In this respect, when
the test-phase attacks are fundamentally different from those observed during
the training phase, the performance of the classical two-class closed-set
systems may be seriously compromised. Due to its significant practical
importance, detection of unseen presentation attacks has received considerable
attention both in face [6, 7, 8, 9, 10, 11, 12, 13] as well as in other
biometric modalities [14, 15, 16, 17, 18].

Although other alternatives may exist, the one-class classification
methodology [19, 20] presents a promising strategy to the unknown face
presentation attack detection problem. Specifically, the undesired training
bias towards seen attacks may be removed by utilising only genuine samples for
one-class training and solely trying to capture the essence of the bona fide
observations. Various studies have looked into the functionality of this
approach to detect unknown/unseen face presentation attacks [7, 6, 21] where
it has been observed that a pure one-class learning approach trained solely on
bona fide samples may yield better generalisation capability against unseen
attacks compared to the closed-set two-class formulations. Although single-
learner one-class face PAD systems have been developed [10, 21, 11, 22],
currently, their capabilities may be insufficient to meet the stringent
requirements of a reliable and secure real-world system.

Among others, ensemble learning serves as a well-known strategy for improving
pattern classification system performance, in which several knowledge sources
for a pattern are merged for the final decision. The variation of this generic
methodology can be related to the level at which the combination is realised,
such as data-level, feature-level, soft-, or hard-decision level, etc. In the
case of face PAD, this approach has been effectively implemented in a pure
one-class setting using a classifier combination [23, 24, 25, 26] or a
multiple kernel fusion [6, 8] scheme with the ensemble system being observed
to outperform each individual learner/representation enrolled in the ensemble.
In particular, resorting to a reproducing kernel Hilbert space (RKHS) [27]
fusion facilitates benefiting from learning strategies with a strong basis in
statistical learning theory [28, 29] while allowing the learning of the
classifier to be separated from the representations. Furthermore, by
constructing each individual kernel based on deep representations, a multiple
kernel system can benefit from deep convolutional representations for improved
detection [8, 6]. As an instance of the kernel-based ensemble methods, the
approach in [8], seeks to infer one-class kernel weights to be fused for a
Fisher null-space classifier in the presence of a matrix-norm regularisation
on kernel weights. An experimental evaluation of the method led to promising
results for detecting unseen attacks. In spite of the relative success of the
one-class ensemble-based techniques for face PAD, one limitation of the
existing methods is that they ignore any inherent variation in the data and
treat the entire observation space similarity by associating global weights to
each representation/learner irrespective of the local region where the pattern
resides. This is an unrealistic assumption since, compared to other
representations/learners, each particular representation/learner may be more
effective in certain regions of the observation space while providing less
discriminative capability in others. As such, a local strategy that considers
potential variability in genuine observations may be advantageous. This aspect
of the problem is overlooked in the existing ensemble one-class face PAD
methods.

In the context of multiple kernel learning (MKL), while the vast majority of
existing algorithms are global in nature, meaning the kernel weights are
shared across the whole observation space, the issue was initially identified
in [30] and has received considerable attention since then [31, 32, 33, 34,
35]. However, existing localised multiple kernel learning methods use non-
convex objective functions which raises questions with regards to their
generalisation capacities. Another potential drawback of a non-convex
formulation is the computational difficulty in obtaining a good local optima,
as well as the ensued hurdles in reproducibility due to variable local optima.
Furthermore, the majority of existing local MKL techniques’ theoretical
performance guarantees are poorly understood. An exception to these is the
study in [36] which formulates the MKL problem as a convex optimisation task.
Yet, certain limitations apply to this method too. First, the MKL problem is
formulated in the presence of an $\ell_{p}$-norm constraint on kernel weights.
More recent studies have demonstrated that a more general matrix
$\ell_{p,q}$-norm regularisation on kernel weights may yield better
performance in different applications settings [8, 37, 34, 38]. Second, the
localised MKL method in [36] decouples the learning tasks into a set of
disjoint MKL problems each associated with a distinct region in the data. Such
an approach completely ignores any dependencies which might exist between
kernel weights corresponding to different clusters. Third, the work in [36]
focuses on a multi-/binary-class SVM-based objective function and does not
provide a straightforward extension for the one-class Fisher null-space
classification, which has demonstrated superior performance for the unknown
face spoofing detection problem.

In this paper, we study the locality information for unknown face PAD in the
context of ensemble techniques and try to capture and use the inherent local
structure in bona fide samples for improved detection performance. More
specifically, drawing on the one-class MKL method of [8], we formulate a local
multiple kernel learning one-class method addressing the aforementioned
limitations of the existing localised MKL approaches and illustrate the
utility of the proposed technique in detecting unknown/unseen face
presentation attacks. In particular, in the proposed method, we regularise
kernel weights using a joint matrix-norm constraint as compared with the
existing local MKL methods using disjoint vector $\ell_{p}$-norm
regularisation. By imposing a joint norm constraint on the collection of
kernel weights from all the existing clusters one enables an active
interaction between kernel weights corresponding to different clusters which
is in sharp contrast with the earlier studies where disjoint regularisation
constraints were considered in the localised methods. Mathematically, we
illustrate that incorporating locality into the MKL framework amounts to
increasing the number of effective kernels (by a factor equal to the number of
clusters) in the ensemble which provides further explanation for its superior
performance as compared with its non-localised variant. We theoretically study
the proposed approach and illustrate its merits in terms of the empirical
Rademacher complexities compared to some other alternatives. Experimentally,
we evaluate the proposed localised one-class MKL algorithm on different
datasets and provide a comparison to the state-of-the-art techniques.

###  I-A Main contributions

A summary of the major contributions of this work is as follows.

  * •

We propose a localised multiple kernel learning method for pure one-class
classification of unknown/unseen face presentation attacks. By virtue of a
localised formulation, the proposed approach is able to capture and benefit
from the intrinsic variability and local structure inherent to bona fide
samples for improved detection performance;

  * •

We formulate the learning problem of the proposed localised MKL approach as a
convex optimisation problem and present an effective method to solve it;

  * •

A theoretical study of the generalisation performance of the proposed method
based on Rademacher complexities is conducted and a comparison against other
alternatives is provided;

  * •

We evaluate the proposed localised MKL one-class approach for general object
image abnormality and novelty detection as well as unseen face PAD and compare
it against state-of-the-art approaches from the literature.

The rest of the paper is structured as detailed next. In Section II, we review
the relevant literature on face PAD with a particular attention to the
unknown/unseen and ensemble face presentation attack detection methods.
Section III briefly reviews the Fisher null projection for one-class
classification and its extension to an MKL setting as presented in [8].
Section IV presents a new localised one-class Fisher null approach and then
extends it to a multiple kernel learning setting followed by an optimisation
approach to solve the corresponding convex optimisation task. Section V
theoretically studies the proposed approach in terms of Rademacher
complexities to derive a generalisation error bound for the proposed method
and theoretically compares the proposed algorithm against some other
alternatives. In Section VI, we present the results of evaluating the proposed
approach on multiple datasets. Finally, Section VII brings the paper to a
conclusion.

##  II Prior Work

Challenge-response, hardware-, and software-based strategies [39, 40, 41, 42]
have all been put into practice to address the face PAD problem, with
software-based alternatives gaining the most traction. Software-based
approaches detect a presentation attack in a video/image by utilising various
intrinsic features of signal content. This work uses a face PAD software
formalism using RGB photos. Regarding cues employed in the software-based
category, texture [43, 44] may be considered as the most extensively deployed
cue for presentation attack detection, but there are other motion-based
methods [45, 46, 47]. Some other studies [48, 49, 50] employ shape or colour
information or spoofing medium contours [51] for face spoofing detection,
while by using frequency information content in the Fourier domain, frequency-
based approaches [52, 53, 47, 54] try to detect presentation attacks. Other
study [55] have employed a statistical method to model noise in the facial PAD
while deep convolutional neural networks serve as a well-known tool for
detecting presentation attacks [56, 57, 58].

From a classification viewpoint, the classical method to detect face
presentation attacks is the closed-set two-class formulation where both
genuine and attack samples are utilised to train the system. Linear
discriminant analysis [48, 59], SVMs [60, 61], neural networks [46] and CNNs
[56, 57], Bayesian networks [62] and Adaboost techniques [63], constitute some
of the most extensively used two-class methods. Regression-based procedures
[64], in contrast, attempt to map the input features to the associated labels.

One-class classification provides a promising alternative to the classical
two-class formulation for face presentation attack detection and has been
found to be especially useful in the case of unseen attacks [7]. As an
instance, the work in [9] employs a Gaussian mixture model to estimate the
distribution of bona fide samples using quality metrics of images. Auto-
Encoder and One-Class SVM are used to detect unknown attacks in [10] while
[21, 11] propose a subject-specific modelling strategy to construct separate
one-class classifiers for each person. Other work [22] considers the unseen
zero-shot face PAD using a tree network while [13] uses a triplet focal
objective function for metric learning.

Ensemble methods, in which the final decision for an object is generated as a
consensus of multiple learners, constitute a relatively successful subclass of
one-class algorithms for detecting unknown face presentation attacks. As an
instance of the work in this group, the study in [23] considers a weighted
linear combination of normalised classifier output scores where the
combination weights are inferred using both negative and positive
observations. However, such a weight adjusting technique raises issues about
the generalisability of the learned weights to unknown attacks. The work in
[24], considers an ensemble of one-class learners fused via a stacking method
and presents a two-stage Genetic Algorithm to improve the generalisation error
by using only genuine samples for training. In a subsequent study [25], the
authors propose to optimise the parameters of a weighted averaging of multiple
classifiers through a two-stage optimisation approach composed of Particle
Swarm and Pattern Search algorithms to minimise the possibility of local
minimum in weight learning. A new classifier score normalisation method for
heavy-tailed distributions is presented to facilitate effective combination of
multiple learners. Evaluation of the algorithm on multiple datasets
illustrated promising performance of this method. However, since the fusion
mechanism in a classifier combination scheme is based solely on the scores
provided by individual classifiers, any extra information that could be
potentially beneficial is omitted. Another study [6] considers a multiple
kernel fusion approach, revealing that a kernel fusion strategy can lead to
substantial improvements in the performance in comparison to each base kernel.
Unlike the classifier fusion paradigm, in a kernel combination method, the
information content related to each representation is not summarised merely in
terms of a single score, allowing the ensemble system to access a more
informative representation of the data. The relative discriminative potential
of each kernel, however, is not taken into account in [6], and all
representations in the composite kernel are weighted equally. Other study [8]
considers a multiple kernel learning algorithm that could capture the inherent
sparsity within a problem and take into account potential discrepancies in the
discriminative content of different kernels and infer kernel weights in a one-
class learning setting to address this shortcoming. The study in [8] presented
a number of desired attributes to build an ensemble face PAD system. First,
instead of classifier scores, it operates on kernel-space representations, and
thus, could access a richer information content for classification. Second,
each representation is weighted in the composite kernel where the weights are
learned using genuine samples only. Third, the multiple kernel learning
approach could minimise the contribution of a potentially less useful
representation to the compound kernel, thanks to deploying a matrix-norm
regularisation constraint inducing sparsity. And last but not the least, it is
formulated as a convex optimisation problem, facilitating optimisation.

Despite their advantages with respect to single-learner PAD systems, the
existing one-class ensemble face PAD techniques have the limitation that they
ignore any inherent variation in the bona fide data and treat the entire
observation space similarly by assigning a global weight to each
representation/learner, regardless of the locale where the model operates.
This is quite a restrictive assumption since each representation/learner may
be more effective in some regions of the observation space while providing
less discriminative capability in others. A local learning strategy may thus
be helpful as it takes into account potential variation in genuine
observations, which is generally neglected in the existing ensemble one-class
face PAD methods. This work addresses this aspect of the ensemble one-class
face presentation attack detection in the context of multiple kernel learning.
By virtue of using local structure, the proposed approach assigns locally
adaptive weights to different representations in distinct regions of the
reproducing kernel Hilbert space and learns the relative importance of the
incorporated representations using a convex formulation.

##  III Preliminaries

The learning problem for the regularised null-space one-class kernel Fisher
method in the primal space is [65]

$\displaystyle\min_{\boldsymbol{\eta}}\frac{\theta}{n}\sum_{i=1}^{n}\big{(}1-\boldsymbol{\psi}(\mathbf{x}_{i})^{\top}\boldsymbol{\eta}\big{)}^{2}+\left\lVert\boldsymbol{\eta}\right\rVert_{2}^{2}$
(1)

where $\mathbf{x}_{i}|_{i=1}^{n}$ denotes the set of positive training
observations and $\boldsymbol{\psi}(\mathbf{x}_{i})$ stands for the feature
associated with $\mathbf{x}_{i}$ in the Hilbert space. The optimal solution to
the problem above yields a linear model in the Hilbert space, i.e.
$f(.)=\boldsymbol{\psi}(.)^{\top}\boldsymbol{\eta}$, that maps observations
onto the regularised Fisher null space for classification. In particular,
positive samples are encouraged to be mapped closer to point $"1"$ while non-
target test observations are expected to be projected further away. In
practice, however, it is more convenient to consider the dual of the problem
in Eq. 1 given as [8, 65]:

$\displaystyle\max_{\boldsymbol{\lambda}}-\boldsymbol{\lambda}^{\top}\mathbf{K}\boldsymbol{\lambda}-\delta\boldsymbol{\lambda}^{\top}\boldsymbol{\lambda}+2\boldsymbol{\lambda}^{\top}\mathbf{1}$
(2)

where $\mathbf{K}$ stands for the kernel matrix, the element of which at the
$j^{th}$ column and $i^{th}$ row is given as
$\mathbf{K}^{i,j}=\boldsymbol{\psi}(\mathbf{x}_{i})^{\top}\boldsymbol{\psi}(\mathbf{x}_{j})=\kappa(\mathbf{x}_{i},\mathbf{x}_{j})$
and $\kappa(.,.)$ stands for the kernel and $\delta=n/\theta$. The optimal
solution to Eq. 2 may be determined by setting the gradient of the objective
function w.r.t. $\boldsymbol{\lambda}$ to zero to obtain
$\boldsymbol{\lambda}=(\mathbf{K}+\delta\mathbf{I})^{-1}\mathbf{1}$, where
$\mathbf{I}$ stands for the identity matrix, and $\mathbf{1}$ is a vector of
$1$’s of dimension $n$. Once the optimal $\boldsymbol{\lambda}$ is computed,
an observation $\mathbf{y}$ is mapped onto the Fisher null-space as

$\displaystyle
f(\mathbf{y})=\Big{[}\kappa(\mathbf{y},\mathbf{x}_{1}),\dots,\kappa(\mathbf{y},\mathbf{x}_{n})\Big{]}\boldsymbol{\lambda}$
(3)

The work in [8] extends the one-class kernel Fisher null method to an MKL
setting by imposing a matrix $\ell_{p,q}$-norm constraint ($p,q\geq 1$) on
kernel weights. Assuming $G$ base kernels $\mathbf{K}_{g}$, $g=1,\dots,G$
which are to be fused linearly using the non-negative weights $\mu_{g}$’s, the
MKL problem considered in [8] is

$\displaystyle\min_{\boldsymbol{\mu}}\max_{\boldsymbol{\lambda}}$
$\displaystyle-\boldsymbol{\lambda}^{\top}\big{(}\sum_{g=1}^{G}\mu_{g}\mathbf{K}_{g}\big{)}\boldsymbol{\lambda}-\delta\boldsymbol{\lambda}^{\top}\boldsymbol{\lambda}+2\boldsymbol{\lambda}^{\top}\mathbf{1}$
(4) $\displaystyle\text{s.t. }\boldsymbol{\mu}\geq
0,\left\lVert\boldsymbol{\mu}\boldsymbol{\mu}^{\top}\right\rVert_{p,q}\leq 1$

where the non-negativity constraint on kernel weights $\mu_{g}$’s,
collectively represented as vector $\boldsymbol{\mu}$, ensures that the
combined kernel matrix is valid while $\left\lVert.\right\rVert_{p,q}$ denotes
a matrix $\ell_{p,q}$-norm ($p,q\geq 1$) defined as

$\displaystyle\left\lVert\boldsymbol{\Upsilon}\right\rVert_{p,q}=\Bigg{(}\sum_{s}\Big{(}\sum_{t}|\upsilon_{ts}|^{p}\Big{)}^{q/p}\Bigg{)}^{1/q}$
(5)

where $s$ and $t$ index the columns and the rows of the matrix, respectively,
and $\upsilon_{ts}$ stands for the element at the $t^{th}$ row and $s^{th}$
column of matrix $\boldsymbol{\Upsilon}$. A matrix mixed-norm constraint on
kernel weights not only provides a controlling mechanism over sparsity of
kernel weights but also enables interactions between kernels [38]. In
contrary, the more commonly used vector $\ell_{p}$-norm regularisation [29]
lacks an explicit mechanism to capture inter-kernel interactions. As such, the
matrix-norm regularisation has been observed to outperform its vector-norm
counterpart in different settings [8, 37, 34, 38]. Note that the vector-norm
constraint may be considered as a special case of the matrix-norm constraint
as setting $p=q$ would reduce the matrix-norm into a vector-norm.

In [8], for solving Eq. 4, $\mathbf{u}$ is assumed as a $G$-dimensional vector
the $g^{th}$ element of which is defined as
$u_{g}\coloneqq\boldsymbol{\lambda}^{\top}\mathbf{K}_{g}\boldsymbol{\lambda}$.
By applying the minimax theorem [66, 67] and switching the order of
optimisation, the problem in Eq. 4 is expressed as

$\displaystyle\max_{\boldsymbol{\lambda}}\Big{\\{}-\delta\boldsymbol{\lambda}^{\top}\boldsymbol{\lambda}+2\boldsymbol{\lambda}^{\top}\mathbf{1}+\min_{\boldsymbol{\mu}}-\boldsymbol{\mu}^{\top}\mathbf{u}\Big{\\}}$
$\displaystyle\text{s.t. }\boldsymbol{\mu}\geq
0,\left\lVert\boldsymbol{\mu}\boldsymbol{\mu}^{\top}\right\rVert_{p,q}\leq 1$
(6)

Next, an iterative two-step method based on an analytical solution in kernel
weights is proposed in [8] to determine the optimisers of the saddle point MKL
problem in Eq. 6.

##  IV The proposed method

In this section, first, we present a novel localised variant of the
regularised null-space Fisher null approach, and then extend the proposed
localised method to a multiple kernel learning setting.

###  IV-A Localised Fisher null projection

In order to incorporate localised information into the one-class kernel Fisher
null method, let us assume that the target/positive training set
$\mathbf{x}_{i}|_{i=1}^{n}$ is probabilistically grouped into $C$ clusters
where $p_{c}(\mathbf{x}_{i})$ denotes the probability of observation
$\mathbf{x}_{i}$ being assigned to the cluster indexed by $c$. Inspired by
[36], we consider a convex combination of $C$ local models each associated
with a different cluster and use cluster likelihoods to form the projection
function. More explicitly, the projection function in the localised model is
defined as

$\displaystyle
f(\mathbf{x}_{i})=\sum_{c=1}^{C}p_{c}(\mathbf{x}_{i})f_{c}(\mathbf{x}_{i})$
(7)

where each local model $f_{c}(.)$ is characterised as
$f_{c}(\mathbf{x}_{i})=\boldsymbol{\psi}(\mathbf{x}_{i})^{\top}\boldsymbol{\eta}_{c}$.
Considering $\boldsymbol{\psi}_{c}(\mathbf{x}_{i})\coloneqq
p_{c}(\mathbf{x}_{i})\boldsymbol{\psi}(\mathbf{x}_{i})$, the primal problem
for the proposed localised one-class kernel Fisher null approach then reads

$\displaystyle\min_{\boldsymbol{\eta}_{c}^{\prime}s}\frac{\theta}{n}\sum_{i=1}^{n}\big{[}1-\sum_{c=1}^{C}\boldsymbol{\psi}_{c}(\mathbf{x}_{i})^{\top}\boldsymbol{\eta}_{c}\big{]}^{2}+\sum_{c=1}^{C}\left\lVert\boldsymbol{\eta}_{c}\right\rVert_{2}^{2}$
(8)

where the first term captures the empirical loss while the second term imposes
a Tikhonov regularisation on the parameters of the localised model.

###  IV-B Dualisation

As it is common in kernel-based methods, we consider the dual of the problem
in Eq. 8. For this purpose, the problem in Eq. 8 can be expressed as

$\displaystyle\min_{\boldsymbol{\eta}_{c}^{\prime}s}$
$\displaystyle\frac{1}{\delta}\left\lVert\boldsymbol{\tau}\right\rVert_{2}^{2}+\sum_{c=1}^{C}\left\lVert\boldsymbol{\eta}_{c}\right\rVert_{2}^{2}$
(9) $\displaystyle\text{s.t.
}\boldsymbol{\tau}=\mathbf{1}-\sum_{c=1}^{C}\boldsymbol{\Psi}_{c}(\mathbf{X})^{\top}\boldsymbol{\eta}_{c}$

where $\boldsymbol{\Psi}_{c}(\mathbf{X})$ denotes a matrix collection of
$\boldsymbol{\psi}_{c}(\mathbf{x}_{i})$’s for $\mathbf{x}_{i}|_{i=1}^{n}$,
$\mathbf{1}$ stands for an $n$-vector of one’s and $\delta=n/\theta$. The
Lagrangian of the constrained problem above is

$\displaystyle\mathcal{L}=\frac{1}{\delta}\left\lVert\boldsymbol{\tau}\right\rVert_{2}^{2}+\sum_{c=1}^{C}\boldsymbol{\eta}_{c}^{\top}\boldsymbol{\eta}_{c}+\boldsymbol{\omega}^{\top}(\mathbf{1}-\sum_{c=1}^{C}\boldsymbol{\Psi}_{c}(\mathbf{X})^{\top}\boldsymbol{\eta}_{c}-\boldsymbol{\tau})$
(10)

where $\boldsymbol{\omega}$ denotes the vector of Lagrange multipliers. Next,
one has to minimise $\mathcal{L}$ in $\boldsymbol{\eta}_{c}$’s and
$\boldsymbol{\tau}$ and then maximise it w.r.t. $\boldsymbol{\omega}$. For the
minimisation of $\mathcal{L}$ w.r.t. $\boldsymbol{\eta}_{c}$’s and
$\boldsymbol{\tau}$, the derivatives with respect to these variables are set
to zero. Omitting the intermediate steps, the Lagrangian is obtained as

$\displaystyle\mathcal{L}=\frac{-1}{4}\boldsymbol{\omega}^{\top}\big{(}\sum_{c=1}^{C}\mathbf{K}_{c}\big{)}\boldsymbol{\omega}-\frac{\delta}{4}\boldsymbol{\omega}^{\top}\boldsymbol{\omega}+\boldsymbol{\omega}^{\top}\mathbf{1}$
(11)

where $\mathbf{K}_{c}$ denotes the kernel matrix associated with cluster $c$
whose $(i,j)^{th}$ element is
$\mathbf{K}_{c}^{i,j}\coloneqq\boldsymbol{\psi}_{c}(\mathbf{x}_{i})^{\top}\boldsymbol{\psi}_{c}(\mathbf{x}_{j})=p_{c}(\mathbf{x}_{i})\kappa(\mathbf{x}_{i},\mathbf{x}_{j})p_{c}(\mathbf{x}_{j})$.
Applying a variable change as $\boldsymbol{\lambda}=\boldsymbol{\omega}/2$,
the dual problem for the proposed localised one-class null-space Fisher
approach is expressed as

$\displaystyle\max_{\boldsymbol{\lambda}}-\boldsymbol{\lambda}^{\top}\big{(}\sum_{c=1}^{C}\mathbf{K}_{c}\big{)}\boldsymbol{\lambda}-\delta\boldsymbol{\lambda}^{\top}\boldsymbol{\lambda}+2\boldsymbol{\lambda}^{\top}\mathbf{1}$
(12)

Setting the gradient of the function above w.r.t $\boldsymbol{\lambda}$ to
zero yields the discriminant in the Hilbert space as
$\boldsymbol{\lambda}=\big{(}\sum_{c=1}^{C}\mathbf{K}_{c}\big{)}^{-1}\mathbf{1}$.
After computing $\boldsymbol{\lambda}$, an observation $\mathbf{y}$ would be
mapped onto the localised Fisher null-space as

$\displaystyle
f(\mathbf{y})=\sum_{c=1}^{C}p_{c}(\mathbf{y})\Big{[}\kappa(\mathbf{y},\mathbf{x}_{1})p_{c}(\mathbf{x}_{1}),\dots,\kappa(\mathbf{y},\mathbf{x}_{n})p_{c}(\mathbf{x}_{n})\Big{]}\boldsymbol{\lambda}$
(13)

As may be observed from Eq. 12, the optimisation problem associated with the
localised approach resembles that of its non-localised counterpart (see Eq. 2)
except for the fact that the effective kernel matrix in the localised model
corresponds to a sum of local kernel matrices. As such, the extensions and
optimisation procedures corresponding to the non-localised approach may be
adapted and applied to the new localised model. In this context, of particular
interest to this study is a multiple kernel learning extension, discussed
next.

###  IV-C Multiple kernel localised Fisher null projection

For the extension of the proposed one-class localised Fisher null approach to
a multiple kernel learning setting, let us assume there exist $G$ base
kernels, $\kappa_{g}(.,.)$, $g=1,\dots,G$ which are to be combined linearly
using non-negative weights. Inspired by the success of a mixed matrix-norm
constraint for multiple kernel one-class classification [8], in this study, we
consider an $\ell_{p,q}$-norm regularisation ($p,q\geq 1$) for kernel weights.
By replacing the kernel matrix of each cluster in Eq. 12 with a weighted
linear fusion of $G$ kernels and imposing matrix-norm and non-negativity
constraints on kernel weights, the learning problem associated with the
proposed localised $\ell_{p,q}$-norm one-class MKL method is defined as

$\displaystyle\min_{\boldsymbol{\mu}}\max_{\boldsymbol{\lambda}}$
$\displaystyle-\boldsymbol{\lambda}^{\top}\big{(}\sum_{c=1}^{C}\sum_{g=1}^{G}\mu_{cg}\mathbf{K}_{cg}\big{)}\boldsymbol{\lambda}-\delta\boldsymbol{\lambda}^{\top}\boldsymbol{\lambda}+2\boldsymbol{\lambda}^{\top}\mathbf{1}$
(14) $\displaystyle\text{s.t. }\boldsymbol{\mu}\geq
0,\left\lVert\boldsymbol{\mu}\boldsymbol{\mu}^{\top}\right\rVert_{p,q}\leq 1$

where $\mathbf{K}_{cg}$ denotes the kernel matrix associated with the kernel
function $\kappa_{g}(.,.)$ for the cluster indexed by $c$. That is, the
$(i,j)^{th}$ element of matrix $\mathbf{K}_{cg}$ is
$\mathbf{K}_{cg}^{i,j}\coloneqq
p_{c}(\mathbf{x}_{i})\kappa_{g}(\mathbf{x}_{i},\mathbf{x}_{j})p_{c}(\mathbf{x}_{j})$.
In Eq. 14, $\boldsymbol{\mu}$ is defined as a $CG$-dimensional vector formed
as the collection of kernel weights from all clusters, i.e.
$\boldsymbol{\mu}=[\boldsymbol{\mu}_{1}^{\top},\dots,\boldsymbol{\mu}_{C}^{\top}]^{\top}$
where $\boldsymbol{\mu}_{c}=[\mu_{c1},\dots,\mu_{cG}]^{\top}$ is the kernel
weight vector associated with cluster $c$. Since $\boldsymbol{\mu}$ is formed
as a collection of kernel weights from all clusters, a norm constraint imposed
on $\boldsymbol{\mu}$ jointly affects the kernel weights corresponding to all
clusters, and thus, promotes interdependence between cluster weights. This is
in contrast to a case where one may impose disjoint constraints on the kernel
weights associated with different clusters [36] which would lack an explicit
interaction mechanism between cluster kernel weights once the data is
clustered. A theoretical analysis regarding the generalisation performance of
the proposed joint norm constraint over all clusters and its advantages
compared to the disjoint case is presented in Section V.

###  IV-D Optimisation

For the optimisation of the saddle point problem in Eq. 14, let us define the
$CG$-element vector $\mathbf{u}$ as
$\mathbf{u}\coloneqq[\mathbf{u}_{1}^{\top},\mathbf{u}_{2}^{\top},\dots,\mathbf{u}_{C}^{\top}]^{\top}$
where the $G$-dimensional vector $\mathbf{u}_{c}$ is

$\displaystyle\mathbf{u}_{c}=[\boldsymbol{\lambda}^{\top}\mathbf{K}_{c1}\boldsymbol{\lambda},\boldsymbol{\lambda}^{\top}\mathbf{K}_{c2}\boldsymbol{\lambda},\dots,\boldsymbol{\lambda}^{\top}\mathbf{K}_{cG}\boldsymbol{\lambda}]$
(15)

Eq. 14 may now be expressed as

$\displaystyle\min_{\boldsymbol{\mu}}\max_{\boldsymbol{\lambda}}$
$\displaystyle-\delta\boldsymbol{\lambda}^{\top}\boldsymbol{\lambda}+2\boldsymbol{\lambda}^{\top}\mathbf{1}-\boldsymbol{\mu}^{\top}\mathbf{u}$
(16) $\displaystyle\text{s.t. }\boldsymbol{\mu}\geq
0,\left\lVert\boldsymbol{\mu}\boldsymbol{\mu}^{\top}\right\rVert_{p,q}\leq 1$

Note that for fixed $\boldsymbol{\lambda}$, the optimisation problem in Eq. 16
is convex w.r.t. $\boldsymbol{\mu}$ since the constraints imposed by the
matrix-norm may be characterised as the supremum of a set of linear functions
of $\boldsymbol{\mu}\boldsymbol{\mu}^{\top}$, and consequently, give rise to a
set which is convex [38, 68]. Furthermore, when $\boldsymbol{\mu}$ is fixed,
the loss is concave w.r.t. $\boldsymbol{\lambda}$. Hence, the minimax theorem
[66, 67] can be utilised to change the optimisation order as

$\displaystyle\max_{\boldsymbol{\lambda}}$
$\displaystyle\Big{\\{}-\delta\boldsymbol{\lambda}^{\top}\boldsymbol{\lambda}+2\boldsymbol{\lambda}^{\top}\mathbf{1}+\min_{\boldsymbol{\mu}}-\boldsymbol{\mu}^{\top}\mathbf{u}\Big{\\}}$
(17) $\displaystyle\text{s.t. }\boldsymbol{\mu}\geq
0,\left\lVert\boldsymbol{\mu}\boldsymbol{\mu}^{\top}\right\rVert_{p,q}\leq 1$

The resulting optimisation problem above has thus become compatible with that
of Eq. 6 of the non-localised MKL approach with the difference that in the
proposed localised method the dimensionality of the kernel weight vector, i.e.
$\boldsymbol{\mu}$, is increased by a factor of number of clusters, i.e. $C$.
Since the structure of the learning problem corresponding to the proposed
localised approach in Eq. 17 matches that of Eq. 6, following a similar
optimisation approach as that advocated in [8], $\boldsymbol{\mu}$ is derived
as (a complete derivation of $\boldsymbol{\mu}$ is provided in the Appendix A)

$\displaystyle\boldsymbol{\mu}=\mathbf{\bar{u}}/\sqrt{\left\lVert\bar{\mathbf{u}}\right\rVert_{p}\left\lVert\bar{\mathbf{u}}\right\rVert_{q}}$

where

$\displaystyle\bar{\mathbf{u}}=\mathbf{u}\odot\Big{(}\boldsymbol{\mu}^{p-2}/\left\lVert\boldsymbol{\mu}\right\rVert_{p}^{p}+\boldsymbol{\mu}^{q-2}/\left\lVert\boldsymbol{\mu}\right\rVert_{q}^{q}\Big{)}^{-1}$
(18)

where $\odot$ denotes a Hadamard (element-wise) multiplication. Once
$\boldsymbol{\mu}$ is calculated, the next step is to maximise the objective
function w.r.t. $\boldsymbol{\lambda}$. This may be readily realised by
following the standard approach to solve for the regularised Fisher null
projection [65] with an effective kernel matrix of
$\mathbf{K}=\sum_{c=1}^{C}\sum_{g=1}^{G}\mu_{cg}\mathbf{K}_{cg}$. Setting the
partial derivative of the cost function w.r.t. $\boldsymbol{\lambda}$ to zero
yields

$\displaystyle\boldsymbol{\lambda}=\Big{(}\delta\mathbf{I}+\sum_{c=1}^{C}\sum_{g=1}^{G}\mu_{cg}\mathbf{K}_{cg}\Big{)}^{-1}\mathbf{1}$
(19)

Algorithm 1 Localised Joint $\ell_{p,q}$-norm Multiple Kernel Null-Space
Fisher Method

1:Estimate $p_{c}(\mathbf{x}_{i})$’s; $i=1,\dots,n$; $c=1,\dots,C$.

2:Construct kernel matrices: $\mathbf{K}_{cg}$’s; $c=1,\dots,C$;
$g=1,\dots,G$.

3:Initialisation:
$\boldsymbol{\lambda}=\big{(}\delta\mathbf{I}+\sum_{c=1}^{C}\sum_{g=1}^{G}(CG)^{\frac{-p-q}{2pq}}\mathbf{K}_{cg}\big{)}^{-1}\mathbf{1}$

4:repeat

5: for $c=1,\dots,C$ do

6:
$\mathbf{u}_{c}=[\boldsymbol{\lambda}^{\top}\mathbf{K}_{c1}\boldsymbol{\lambda},\boldsymbol{\lambda}^{\top}\mathbf{K}_{c2}\boldsymbol{\lambda},\dots,\boldsymbol{\lambda}^{\top}\mathbf{K}_{cG}\boldsymbol{\lambda}]^{\top}$

7: end for

8:
$\mathbf{u}=[\mathbf{u}_{1}^{\top},\mathbf{u}_{2}^{\top},\dots,\mathbf{u}_{C}^{\top}]^{\top}$

9:
$\bar{\mathbf{u}}=\mathbf{u}\odot\Big{(}\boldsymbol{\mu}^{p-2}/\left\lVert\boldsymbol{\mu}\right\rVert_{p}^{p}+\boldsymbol{\mu}^{q-2}/\left\lVert\boldsymbol{\mu}\right\rVert_{q}^{q}\Big{)}^{-1}$

10:
$\boldsymbol{\mu}=\mathbf{\bar{u}}/\sqrt{\left\lVert\bar{\mathbf{u}}\right\rVert_{p}\left\lVert\bar{\mathbf{u}}\right\rVert_{q}}$

11:
$\boldsymbol{\lambda}=\Big{(}\delta\mathbf{I}+\sum_{c=1}^{C}\sum_{g=1}^{G}\mu_{cg}\mathbf{K}_{cg}\Big{)}^{-1}\mathbf{1}$

12:until convergence

13:Output: $\boldsymbol{\lambda}$ and $\boldsymbol{\mu}$

Note that the relation above gives $\boldsymbol{\lambda}$ in terms of
$\boldsymbol{\mu}$ which is itself a function of $\boldsymbol{\lambda}$.
Denoting
$\Big{(}\delta\mathbf{I}+\sum_{c=1}^{C}\sum_{g=1}^{G}\mu_{cg}\mathbf{K}_{cg}\Big{)}^{-1}\mathbf{1}\coloneqq
h(\boldsymbol{\lambda})$, the optimal $\boldsymbol{\lambda}$ should then
fulfil $\boldsymbol{\lambda}=h(\boldsymbol{\lambda})$. In order to determine
$\boldsymbol{\lambda}$ satisfying
$\boldsymbol{\lambda}=h(\boldsymbol{\lambda})$, a fixed-point iteration method
[69] is applied. Algorithm 1 summarises the approach described above where
$\boldsymbol{\lambda}$ is initialised using uniform kernel weights with a unit
$\ell_{p,q}$-norm. Once $\boldsymbol{\lambda}$ and $\boldsymbol{\mu}$ are
computed, an observation $\mathbf{y}$ can be mapped onto the localised Fisher
null subspace as

$\displaystyle
f(\mathbf{y})=\sum_{c=1}^{C}p_{c}(\mathbf{y})\sum_{g=1}^{G}\mu_{cg}\Big{[}\kappa_{g}(\mathbf{y},\mathbf{x}_{1})p_{c}(\mathbf{x}_{1}),$
$\displaystyle\dots,\kappa_{g}(\mathbf{y},\mathbf{x}_{n})p_{c}(\mathbf{x}_{n})\Big{]}\boldsymbol{\lambda}$
(20)

The following proposition characterises the convergence properties of the
proposed localised matrix-norm multiple kernel learning approach summarised in
Algorithm 1.

###### Proposition 1

For an adequately large $\delta$, the proposed approach will converge to a
single point independent of a particular initilisation for
$\boldsymbol{\lambda}$.

Consult Appendix B for a proof.

##  V Theoretical Analysis

In this section, a theoretical analysis of the proposed localised
$\ell_{p,q}$-norm one-class MKL approach based on Rademacher complexities is
conducted which facilitates deriving a generalisation error bound for the
proposed method. Furthermore, the derived Rademacher complexity bound enables
a theoretical comparison between the proposed approach and some other
alternative methods.

The hypothesis set for the proposed localised $\ell_{p,q}$-norm one-class MKL
method with $C$ clusters and $G$ kernels may be expressed as

$\displaystyle
H_{CG}^{p,q}=\Big{\\{}f:\mathcal{X}\rightarrow\mathbb{R}\Big{|}f(\mathbf{y})=\sum_{c}p_{c}(\mathbf{y})\sum_{g}\mu_{cg}\big{[}\kappa_{g}(\mathbf{y},\mathbf{x}_{1})$
$\displaystyle
p_{c}(\mathbf{x}_{1}),\dots,\kappa_{g}(\mathbf{y},\mathbf{x}_{n})p_{c}(\mathbf{x}_{n})\big{]}\boldsymbol{\lambda},$
$\displaystyle\boldsymbol{\lambda}^{\top}\big{(}\sum_{c}\sum_{g}\mu_{cg}\mathbf{K}_{cg}\big{)}\boldsymbol{\lambda}\leq\Lambda^{2},\left\lVert\boldsymbol{\mu}\boldsymbol{\mu}^{\top}\right\rVert_{p,q}\leq
1\Big{\\}}$

where the constraint
$\boldsymbol{\lambda}^{\top}\big{(}\sum_{c}\sum_{g}\mu_{cg}\mathbf{K}_{cg}\big{)}\boldsymbol{\lambda}\leq\Lambda^{2}$
follows from the equivalency between a Tikhonov and an Ivanov regularisation
for some $\Lambda$ [29]. The empirical Rademacher complexity [70] of the
hypothesis class $H_{CG}^{p,q}$ for samples $\mathbf{x}_{i}$’s is

$\displaystyle\mathcal{R}(H_{CG}^{p,q})=\frac{1}{n}\mathbb{E}_{\boldsymbol{\sigma}}\Big{[}\mathrm{sup}_{f\in
H_{CG}^{p,q}}\sum_{i}\sigma_{i}f(\mathbf{x}_{i})\Big{]}$ (22)

where $\sigma_{i}$’s are uniform independent random variables with values in
$\\{-1,+1\\}$ called Rademacher variables and $\boldsymbol{\sigma}$ is a
vector collection of $\sigma_{i}$’s. The Rademacher complexity measures the
richness of the function class $H_{CG}^{p,q}$ by quantifying how well the
class of functions under analysis is capable of correlating with random
labels.

###### Proposition 2

The empirical Rademacher complexity of the proposed approach is bounded as

$\displaystyle\mathcal{R}(H_{CG}^{p,q})\leq\frac{\Lambda
r}{n}(CG)^{\frac{1}{2}-\frac{1}{4p}-\frac{1}{4q}}\Bigg{(}\sum_{c}\sum_{i}p_{c}^{2}(\mathbf{x}_{i})\Bigg{)}^{1/2}$
(23)

A proof is provided in Appendix C.

Using the Rademacher complexity, one may characterise the generalisation
performance of the proposed algorithm as follows.

######  Proposition 3 (Generalisation error bound)

Assuming that the kernel function is bounded by $r^{2}$ and the squared error
loss is bounded by $B_{l}$ and is $\mu$-Lipschitz for some $\mu$, then the
following inequality holds with confidence larger than $1-\Delta$ over samples
of size $n$ for all classifiers $f\in\mathcal{H}_{CG}^{p,q}$

$\displaystyle\varepsilon(f)\leq\frac{1}{n}\sum_{i}(f(\mathbf{x}_{i})-1)^{2}+$
$\displaystyle\frac{\Lambda
r}{n}(CG)^{\frac{1}{2}-\frac{1}{4p}-\frac{1}{4q}}\Bigg{(}\sum_{c}\sum_{i}p_{c}^{2}(\mathbf{x}_{i})\Bigg{)}^{1/2}+3B_{l}\sqrt{\frac{\ln(2/\Delta)}{2n}}$
(24)

where
$\varepsilon(f)\coloneqq\mathbb{E}\Big{[}(\hat{f}(\mathbf{y})-f(\mathbf{y}))^{2}\Big{]}$,
and $\hat{f}(.)$ denotes the true label of an observation.

Proof The proof follows by plugging in the Rademacher complexity bound and the
empirical loss function of the proposed approach into Theorem 11.3 in
[70].$\square$

The generalisation error bound in Eq. 24 suggests a trade-off between reducing
the empirical error over the training set and controlling the Rademacher
complexity of $\mathcal{H}_{CG}^{p,q}$. As suggested by Eq. 23, the empirical
Rademacher complexity for the hypothesis set $\mathcal{H}_{CG}^{p,q}$ may be
controlled using $\Lambda$ which depends on $\delta$ (see Appendix D for a
proof). As a result, in order to optimise the generalisation error bound, one
needs to set $\theta$ (or equivalently $\delta$) so as to maintain a trade-off
between different terms in the upper bound given in Eq. 24. In practice, a
validation set independent from the training set may be used to tune $\theta$
so that a performance metric is optimised.

###  V-A Other regularisation constraints

In this study, we considered a joint $\ell_{p,q}$-norm constraint on the
collection of kernel weights from all the existing clusters. Other closely
related alternatives for norm regularisation include: (1) a joint vector
$\ell_{p}$-norm; (2) a disjoint vector $\ell_{p}$-norm; or (3) a disjoint
matrix $\ell_{p,q}$-norm. For the joint vector $\ell_{p}$-norm regularisation,
the constraint is expressed as
$\left\lVert\boldsymbol{\mu}\right\rVert_{p}\leq 1$ while for the disjoint
vector-norm regularisation, the constraint set is defined as
$\left\lVert\boldsymbol{\mu}_{c}\right\rVert_{p}\leq 1$, for $c=1,\dots,C$.
Regarding the disjoint $\ell_{p,q}$-norm regularisation, the constraints are
$\left\lVert\boldsymbol{\mu}_{c}\boldsymbol{\mu}_{c}^{\top}\right\rVert_{p,q}\leq
1$, for $c=1,\dots,C$. As formally stated in the following proposition, the
empirical Rademacher complexity bound of the proposed joint matrix-norm
approach is lower than those of the joint vector-norm, disjoint vector-norm,
and the disjoint matrix-norm class.

###### Proposition 4

The empirical Rademacher complexity of the proposed localised joint
$\ell_{p,q}$-norm ($1\leq q\leq p$) one-class MKL approach is

  1. (i)

lower than the localised joint vector-norm function class by a factor of
$(CG)^{\frac{1}{4p}-\frac{1}{4q}}$

  2. (ii)

lower than the localised disjoint vector-norm function class by a factor of
$C^{-\frac{1}{4p}-\frac{1}{4q}}G^{\frac{1}{4p}-\frac{1}{4q}}$,

  3. (iii)

lower than the localised disjoint matrix-norm function class by a factor of
$C^{-\frac{1}{4p}-\frac{1}{4q}}$

A proof is provided in Appendix E.

###  V-B Runtime Complexity

During training, we incur $\mathcal{O}(CGn^{2})$ for computing kernel
matrices, and during each training iteration $\mathcal{O}(Cn^{2})$ for
computing vector $\mathbf{u}$ while for solving the linear equation
$\Big{(}\delta\mathbf{I}+\sum_{c=1}^{C}\sum_{g=1}^{G}\mu_{cg}\mathbf{K}_{cg}\Big{)}\boldsymbol{\lambda}=\mathbf{1}$
via incremental Cholesky factorisation using Sherman’s March algorithm [71]
and forward-back substitution we need $\mathcal{O}(n^{2})$. The complexity at
the test phase is $\mathcal{O}(CGn)$.

##  VI Experiments and Results

In this section, the results of evaluating the proposed approach on different
datasets are presented and compared against existing methods. The rest of this
section is structured as detailed below.

  * •

Section VI-A, provides the details of our implementation.

  * •

In Section VI-B, we visualise the local kernel weights inferred for some
sample classes to confirm the possibility of inferring locally adaptive kernel
weights using the proposed method.

  * •

In Section VI-C, the results of evaluating the proposed approach on one-class
general object classification for abnormality and novelty detection on the
Caltech256 [72] and the 1001 abnormal objects [73] datasets are presented and
compared against other methods.

  * •

Section VI-D presents the results of the unseen face PAD on four widely used
datasets in a zero-shot evaluation scenario and compares the proposed approach
with the state-of-the-art techniques.

Figure 1: Sample local weights for seven kernels in 3 different clusters for
two sample classes (left and right columns) from the Caltech256 dataset.

###  VI-A Details of implementation

In the following experiments, we use an RBF kernel function and set its width
to half of the mean pairwise Euclidean distance over training observations.
$\delta$ is selected from $\\{10^{-3},10^{-2},10^{-1},1,10,10^{2}\\}\times n$
while $p$ and $q$ are selected from $\\{32/31,16/15,8/7,4/3,2,4,8,10\\}$. On
all datasets, the parameters above are set using the data from classes other
than the target class under consideration. The number of genuine data clusters
$C$ is set to 3 in all the following experiments. We form the clusters using
the kernel k-means algorithm and derive $p_{c}(\mathbf{x}_{i})$’s using a
softmax function operating on the distances from cluster centres.

###  VI-B Effect of locality

The objective of the current study is to infer locally adaptive kernel
weights. In order to visualise and confirm that different weights can be
inferred for different data clusters, we randomly pick two classes from the
Caltech256 dataset [72], and following [8], construct seven kernels using deep
representations derived from the pre-trained convolutional neural networks of
Resnet50 [74], Googlenet [75], Alexnet [76], Vgg16 [77], Densenet201 [78],
Mobilenetv2 [79], and Nasnetlarge [80]. We set the number of clusters to three
and run the proposed algorithm to infer local kernel weights. The weights
inferred for the seven kernel matrices used are visualised in Fig. 1. As it
can be seen from the figure, for both object classes, a specific kernel can be
weighted differently in different clusters. As such, the proposed approach is
effective is inferring locally adaptive kernel weights. The utility of the
inferred local weights for one-class classification is analysed in the
following sections.

###  VI-C One-class general object classification

In the first set of experiments, the proposed approach is evaluated for one-
class classification of general object images. For this purpose, two
experiments for novelty and abnormality detection are conducted on the
Caltech256 and 1001 abnormal objects datasets, discussed next.

####  VI-C1 Novelty detection

The Caltech 256 dataset [72] is commonly used to test novelty detection
algorithms. The dataset incorporates images of objects from 256 different
classes giving rise to 30607 images in total. For novelty detection, different
protocols have been introduced to convert the multi-class dataset into one
suitable for one-class novelty detection. So as to conduct a fair comparison
with other approaches, in this work, we follow one of the widely used
evaluation protocols introduced in [81] where a single class is presumed as
the target/normal class and the learned model is then evaluated by considering
objects from all the remaining 255 classes as novel objects. The experiment is
repeated for the first 40 classes of the database.

####  VI-C2 Abnormality detection

For abnormality detection, one of the frequently employed image datasets is
the 1001 Abnormal Objects database [73]. The dataset includes 1001 image
samples of six different object classes from the PASCAL database [82]. The
goal on this database is to classify each image as either normal or abnormal.
While the abnormal samples correspond to 6 different categories and are
annotated using Amazon Mechanical Turk, the normal instances are obtained from
the PASCAL dataset from each corresponding object class.

On the Caltech256 and 1001 Abnormal object databases, we use the generally
known AUC performance metric to compare the performance of the proposed method
to those of existing methods. The AUC provides a performance statistic
independent of a particular threshold by measuring the area under the Receiver
Operating Characteristic curve (ROC), which contrasts the true positive rate
against the false positive rate at varying thresholds. In order to
specifically gauge the performance benefits brought by the proposed localised
MKL approach compared to the non-localised matrix-regularised MKL [8], we use
the features obtained from the seven deep pre-trained CNN’s of Resnet50 [74],
Googlenet [75], Alexnet [76], Vgg16 [77], Densenet201 [78], Mobilenetv2 [79],
and Nasnetlarge [80] as suggested in [8] to construct $21$ localised kernel
matrices ($7$ representations $\times$ $3$ clusters) and compare the
performance of the proposed method to its non-localised variant as well as
other methods from the literature. The results of this set of experiments are
tabulated in Table I and Table II for the Caltech256 and 1001 Abnormal Objects
datasets, respectively. From the tables, the following observations may be
made. On the Caltech256 dataset, while the non-localised MKL approach achieves
an average AUC of $99.6\%$, the proposed localisation improves this
performance and yields a perfect classification for novelty detection, ranking
the best for novelty detection on the Caltech256 database. On the 1001
Abnormal Objects database, while the average AUC of the non-localised method
is $99.2\%$, the proposed localised MKL approach obtains an average AUC of
$99.4\%$, performing better than other alternatives.

TABLE I: Performances of different methods on the Caltech256 database for novelty detection in terms of AUC (MEAN$\pm$STD$\%$). Approach | AUC
---|---
OCNN-AlexNet [83] | $82.6\pm 15.3$
OCNN-VGG16 [83] | $88.5\pm 14.4$
Autoencoder [84] | $62.3\pm 7.2$
OCSVM-VGG16 [85] | $90.2\pm 5.0$
DOC-VGG16 [85] | $98.1\pm 2.2$
DOC-AlexNet [85] | $95.9\pm 2.1$
Multiple Kernel-SVDD [8] | $99.1\pm 0.7$
Multiple Kernel-SVDD-Slim [8] | $99.2\pm 0.6$
Multiple Kernel-OCSVM [8] | $99.1\pm 0.8$
Multiple Kernel-OCSVM-Slim [8] | $99.1\pm 0.8$
$\ell_{p,q}$-MKL [8] | $99.6\pm 0.5$
Localised MKL (this work) | $\mathbf{100\pm 0}$
TABLE II: Performances of different methods on the Abnormality-1001 database for abnormality detection in terms of AUC (MEAN$\pm$STD$\%$). Approach | AUC
---|---
Graphical Model [73] | $87.0\pm n.a.$
Adj. Graphical Model [73] | $91.1\pm n.a.$
OCNN [83] | $88.5\pm 1.4$
Autoencoder [84] | $67.4\pm 1.2$
OC-CNN [86] | $84.3\pm n.a.$
DOC [85] | $95.6\pm 3.1$
Multiple Kernel-SVDD [8] | $92.3\pm 0.2$
Multiple Kernel-SVDD-Slim [8] | $94.1\pm 0.1$
Multiple Kernel-OCSVM [8] | $91.4\pm 0.3$
Multiple Kernel-OCSVM-Slim [8] | $93.9\pm 0.1$
$\ell_{p,q}$-MKL [8] | $99.2\pm 0.2$
Localised MKL (this work) | $\mathbf{99.4\pm 0.4}$

###  VI-D One-class unseen face PAD

In this set of experiments, the proposed approach is evaluated for the unknown
face PAD and compared to other methods that follow a similar zero-shot unseen
PA detection setting. For this purpose, we use four commonly used face PAD
databases briefly introduced below.

####  VI-D1 The Replay-Mobile database [87]

incorporates 1190 video sequences of genuine and attack observations from 40
subjects that are captured by two different video cameras under various
lighting conditions. In this dataset, three disjoint subsets exist which are
to be used for training, development, and testing purposes as well as an
additional enrolment subset.

####  VI-D2 The OULU-NPU database [88]

includes 4950 attack and genuine videos corresponding to 55 individuals
recorded with 6 different cameras in 3 sessions subject to various lighting
conditions and background settings. The dataset evaluation protocol requires
PA detection subject to previously unseen video acquisition settings, unseen
attack types and also unseen camera sensors. The videos are separated into
three subject-disjoint groups as train, development, and test sets. For
testing, 4 evaluation protocols are considered among which Protocol IV is the
most difficult one and is followed in this work.

####  VI-D3 The MSU-MFSD dataset [89]

incorporates 440 videos recorded from either video or photo attack trials
corresponding to 55 individuals which are captured using 2 different devices.
The subset of the dataset which is publicly available, includes videos
corresponding to 35 subjects. The dataset is partitioned to 2 sets for
training and testing purposes with no overlap between the subjects from the
two sets.

####  VI-D4 The Replay-Attack dataset [90]

incorporates 1300 videos of genuine and attack data from fifty subjects.
Attack data are generated by means of a mobile phone, an HD iPad screen, or
using a printed image. The database is randomly divided to three subject-
disjoint partitions for training, development, and testing purposes.

The ISO metrics [91] used to gauge presentation attack detection system
performance are: 1) BPCER: bona fide presentation classification error rate
that measures the ratio of genuine samples which are incorrectly classified;
and 2) APCER: attack presentation classification error rate that reflects the
ratio of incorrectly classified PA’s realised with a similar PA instrument
species. Based on the maximum APCER over all attack instruments, the
presentation attack detection system performance may be summarised in terms of
ACER, i.e. the Average Classification Error Rate as

$\displaystyle ACER=\Big{(}BPCER+\max_{PAIS}APCER_{PAIS}\Big{)}/2$ (25)

In addition to the aforementioned metrics and to enable a comparison of the
performance of the proposed method to other approaches, we also report the
performance of the proposed method in terms of the Area Under the ROC Curve
(AUC) and Half Total Error Rate (HTER). The kernels used in this set of
experiments are those suggested in [8] which have been observed to be
effective for face presentation attack detection. These kernels correspond to
deep CNN features extracted using Resnet50 [74], Googlenet [75], and Vgg16
[77] pre-trained models from four different facial regions, giving rise to
$36$ kernels in the proposed localised approach:
$3\mathrm{(CNN^{\prime}s)}\times 4\mathrm{(regions)}\times
3\mathrm{(clusters)}$. Using similar deep representations as those suggested
in [8] enables a direct comparison between the proposed localised MKL method
and its non-localised variant. The results of this set of experiments are
provided in Table III, Table IV, Table V, and Table VI, for the Replay-Mobile
[87], Oulu-NPU [88], MSU-MFSD [89], and Replay-Attack [90] databases,
respectively. The following observations from the tables are in order. On the
Replay-Mobile database, in an unknown face PAD setting, the non-localised MKL
method of [8] obtains a HTER of $6.7\%$. The proposed localised MKL approach
reduces the error rate to $5.6\%$ accounting to more than $15\%$ improvement
in terms of HTER. The proposed localised approach also outperforms other
multiple kernel learning systems operating on SVM formulations. The proposed
approach also yields a better performance compared to the weighted classifier
fusion method presented in [23] and also better than the method of [21]. On
the Oulu-NPU protocol IV, a similar observation may be made. That is, the
proposed localised learning approach improves upon its non-localised variant.
In particular, while the average APCER of the proposed approach is $3.3\%$,
for the non-localised version the corresponding average APCER is $5.0$ which
amounts to more than $30\%$ improvement in terms of APCER. The overall
performance of the proposed method on the Oulu-NPU dataset measured as ACER is
better than other methods evaluated in an unknown face spoofing detection
scenario. For the MSU-MFSD and the Replay-Attack databases, the proposed
approach provides a perfect separation between bona fide and attack samples as
may be observed from the corresponding tables.

TABLE III: Performances of different methods in terms of Half Total Error Rate ($\%$) for the unknown face PAD on the Replay-Mobile database. Approach | HTER
---|---
The work in [21] | $8.5$
Weighted-Fusion [23] | $9.9$
GoogleNet+MD [11] | $13.7$
The work in [6] | $11.8$
Multiple Kernel-SVDD [8] | $11.1$
Multiple Kernel-SVDD-Slim [8] | $7.6$
Multiple Kernel-OCSVM [8] | $10.8$
Multiple Kernel-OCSVM-Slim [8] | $8.1$
$\ell_{p,q}$-MKL [8] | $6.7$
Localised MKL (this work) | $\mathbf{5.6}$
TABLE IV: Performances of different methods in terms of BPCER, APCER, and ACER ($\%$) for the unknown face PAD on the Oulu-NPU database protocol IV (mean$\pm$std $\%$). Approach | BPCER | APCER | ACER
---|---|---|---
SAPLC [92] | $6.6\pm 5.5$ | $11.9\pm 6.9$ | $9.3\pm 4.4$
OCA-FAS [93] | $5.9\pm 4.5$ | $2.3\pm 2.5$ | $4.1\pm 2.7$
The work in [94] | $1.7\pm 2.6$ | $5.8\pm 4.9$ | $3.7\pm 2.1$
The work in [95] | $9.7\pm 4.8$ | $11.3\pm 3.9$ | $9.8\pm 4.2$
The work in [6] | $0.8\pm 2.0$ | $11.6\pm 13.6$ | $6.2\pm 6.8$
Multiple Kernel-SVDD [8] | $2.5\pm 4.2$ | $11.7\pm 10.8$ | $7.1\pm 6.2$
Multiple Kernel-SVDD-Slim [8] | $1.7\pm 4.1$ | $10.8\pm 8.0$ | $6.2\pm 4.4$
Multiple Kernel-OCSVM [8] | $2.5\pm 4.2$ | $13.3\pm 11.7$ | $7.9\pm 6.4$
Multiple Kernel-OCSVM-Slim [8] | $1.7\pm 4.1$ | $10.8\pm 8.0$ | $6.2\pm 4.4$
$\ell_{p,q}$-MKL [8] | $0\pm 0$ | $5.0\pm 4.5$ | $2.5\pm 2.2$
Localised MKL (this work) | $0\pm 0$ | $3.3\pm 4.1$ | $\mathbf{1.6\pm 2.0}$
TABLE V: Performances of different methods in terms of AUC ($\%$) for the unknown face PAD on on the MSU-MFSD database. Method | AUC
---|---
IMQ+OCSVM [7] | $67.7$
BSIF+OCSVM [7] | $75.6$
LBP+NN [10] | $81.6$
LBP+GMM [10] | $81.3$
LBP+OCSVM [10] | $84.5$
LBP+AE [10] | $87.6$
DTL [12] | $93.0$
The work in [6] | $\mathbf{100}$
Multiple Kernel-SVDD [8] | $\mathbf{100}$
Multiple Kernel-SVDD-Slim [8] | $\mathbf{100}$
Multiple Kernel-OCSVM [8] | $\mathbf{100}$
Multiple Kernel-OCSVM-Slim [8] | $\mathbf{100}$
$\ell_{p,q}$-MKL [8] | $\mathbf{100}$
Localised MKL (this work) | $\mathbf{100}$
TABLE VI: Performances of different methods in terms AUC ($\%$) for the unknown face PAD on on the Replay-Attack database. Method | AUC
---|---
IMQ+OCSVM [7] | $80.7$
BSIF+OCSVM [7] | $81.9$
LBP+NN [10] | $91.2$
LBP+GMM [10] | $90.1$
LBP+OCSVM [10] | $87.9$
LBP+AE [10] | $86.1$
DTL [12] | $99.8$
MD [11] | $99.7$
The work in [6] | $\mathbf{100}$
Multiple Kernel-SVDD [8] | $\mathbf{100}$
Multiple Kernel-SVDD-Slim [8] | $\mathbf{100}$
Multiple Kernel-OCSVM [8] | $\mathbf{100}$
Multiple Kernel-OCSVM-Slim [8] | $\mathbf{100}$
$\ell_{p,q}$-MKL [8] | $\mathbf{100}$
Localised MKL (this work) | $\mathbf{100}$

##  VII Conclusion

We considered the unseen face PAD problem in the context of one-class multiple
kernel learning. While earlier MKL studies failed to explicitly take into the
variability in bona fide samples in learning kernel weights, we presented a
localised one-class MKL algorithm which provided locally adaptive kernel
weights. We formulated the corresponding problem as a convex optimisation task
and presented an effective method for its optimisation. The results of
evaluating the proposed method on general object image and face spoofing data
verified its efficacy in detection presentation attacks by solely training on
genuine samples while a theoretical analysis of the proposed localised
multiple kernel learning approach based on Rademacher complexities illustrated
its benefits compared to some other alternatives.

## Acknowledgements

This research is supported by The Scientific and Technological Research
Council of Turkey (TÜBİTAK) under the grant no 121E465.

## References

  * [1] C. Low, A. B. Teoh, and C. Ng, “Multi-fold gabor, pca, and ica filter convolution descriptor for face recognition,” _IEEE Transactions on Circuits and Systems for Video Technology_ , vol. 29, no. 1, pp. 115–129, 2019\.
  * [2] A. Sepas-Moghaddam, M. A. Haque, P. L. Correia, K. Nasrollahi, T. B. Moeslund, and F. Pereira, “A double-deep spatio-angular learning framework for light field based face recognition,” _IEEE Transactions on Circuits and Systems for Video Technology_ , pp. 1–1, 2019.
  * [3] H. Cevikalp, H. S. Yavuz, and B. Triggs, “Face recognition based on videos by using convex hulls,” _IEEE Transactions on Circuits and Systems for Video Technology_ , pp. 1–1, 2019.
  * [4] S. Yang, W. Deng, M. Wang, J. Du, and J. Hu, “Orthogonality loss: Learning discriminative representations for face recognition,” _IEEE Transactions on Circuits and Systems for Video Technology_ , pp. 1–1, 2020.
  * [5] W. J. Scheirer, A. de Rezende Rocha, A. Sapkota, and T. E. Boult, “Toward open set recognition,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 35, no. 7, pp. 1757–1772, 2013.
  * [6] S. R. Arashloo, “Unseen face presentation attack detection using sparse multiple kernel fisher null-space,” _IEEE Transactions on Circuits and Systems for Video Technology_ , vol. 31, no. 10, pp. 4084–4095, 2021.
  * [7] S. R. Arashloo, J. Kittler, and W. Christmas, “An anomaly detection approach to face spoofing detection: A new formulation and evaluation protocol,” _IEEE Access_ , vol. 5, pp. 13 868–13 882, 2017.
  * [8] S. R. Arashloo, “Matrix-regularized one-class multiple kernel learning for unseen face presentation attack detection,” _IEEE Transactions on Information Forensics and Security_ , vol. 16, pp. 4635–4647, 2021.
  * [9] O. Nikisins, A. Mohammadi, A. Anjos, and S. Marcel, “On effectiveness of anomaly detection approaches against unseen presentation attacks in face anti-spoofing,” in _2018 International Conference on Biometrics (ICB)_ , Feb 2018, pp. 75–81.
  * [10] F. Xiong and W. AbdAlmageed, “Unknown presentation attack detection with face rgb images,” in _2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS)_ , Oct 2018, pp. 1–9.
  * [11] S. Fatemifar, S. R. Arashloo, M. Awais, and J. Kittler, “Spoofing attack detection by anomaly detection,” in _ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , May 2019, pp. 8464–8468.
  * [12] Y. Liu, J. Stehouwer, A. Jourabloo, and X. Liu, “Deep tree learning for zero-shot face anti-spoofing,” in _2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2019, pp. 4675–4684.
  * [13] D. Pérez-Cabo, D. Jiménez-Cabello, A. Costa-Pazo, and R. López-Sastre, “Deep anomaly detection for generalized face anti-spoofing,” in _2019 IEEE workshop on Computer Vision and Pattern Recognition_ , 04 2019.
  * [14] A. Rattani and A. Ross, “Automatic adaptation of fingerprint liveness detector to new spoof materials,” in _IEEE International Joint Conference on Biometrics_ , Sep. 2014, pp. 1–8.
  * [15] A. Rattani, W. J. Scheirer, and A. Ross, “Open set fingerprint spoof detection across novel fabrication materials,” _IEEE Transactions on Information Forensics and Security_ , vol. 10, no. 11, pp. 2447–2460, Nov 2015\.
  * [16] Y. Ding and A. Ross, “An ensemble of one-class svms for fingerprint spoof detection across different fabrication materials,” in _2016 IEEE International Workshop on Information Forensics and Security (WIFS)_ , Dec 2016, pp. 1–6.
  * [17] P. M. Ferreira, A. F. Sequeira, D. Pernes, A. Rebelo, and J. S. Cardoso, “Adversarial learning for a robust iris presentation attack detection method against unseen attack presentations,” in _2019 International Conference of the Biometrics Special Interest Group (BIOSIG)_ , Sep. 2019, pp. 1–7.
  * [18] M. Sahidullah, H. Delgado, M. Todisco, T. Kinnunen, N. Evans, J. Yamagishi, and K.-A. Lee, _Introduction to Voice Presentation Attack Detection and Recent Advances_. Cham: Springer International Publishing, 2019, pp. 321–361.
  * [19] S. S. Khan and M. G. Madden, “One-class classification: taxonomy of study and review of techniques,” _The Knowledge Engineering Review_ , vol. 29, no. 3, p. 345–374, 2014.
  * [20] D. M. Tax and R. P. Duin, “Support vector data description,” _Machine Learning_ , vol. 54, no. 1, pp. 45–66, Jan 2004.
  * [21] S. Fatemifar, S. R. Arashloo, M. Awais, and J. Kittler, “Client-specific anomaly detection for face presentation attack detection,” _Pattern Recognition_ , vol. 112, p. 107696, 2021.
  * [22] A. J. Yaojie Liu, Joel Stehouwer and X. Liu, “Deep tree learning for zero-shot face anti-spoofing,” _CoRR_ , vol. abs/1904.02860, 2019.
  * [23] S. Fatemifar, M. Awais, S. R. Arashloo, and J. Kittler, “Combining multiple one-class classifiers for anomaly based face spoofing attack detection,” in _2019 International Conference on Biometrics (ICB)_ , 2019, pp. 1–7.
  * [24] S. Fatemifar, M. Awais, A. Akbari, and J. Kittler, “A stacking ensemble for anomaly based client-specific face spoofing detection,” in _2020 IEEE International Conference on Image Processing (ICIP)_ , 2020, pp. 1371–1375.
  * [25] ——, “Particle swarm and pattern search optimisation of an ensemble of face anomaly detectors,” in _2021 IEEE International Conference on Image Processing (ICIP)_ , 2021, pp. 3622–3626.
  * [26] S. Fatemifar, S. Asadi, M. Awais, A. Akbari, and J. Kittler, “Face spoofing detection ensemble via multistage optimisation and pruning,” _Pattern Recognition Letters_ , 2022.
  * [27] C. Cortes and V. Vapnik, “Support-vector networks,” _Machine Learning_ , vol. 20, no. 3, pp. 273–297, Sep 1995.
  * [28] V. N. Vapnik, _Statistical Learning Theory_. Wiley-Interscience, 1998.
  * [29] M. Kloft, U. Brefeld, S. Sonnenburg, and A. Zien, “$\ell_{p}$-norm multiple kernel learning,” _Journal of Machine Learning Research_ , vol. 12, no. 26, pp. 953–997, 2011. [Online]. Available: http://jmlr.org/papers/v12/kloft11a.html
  * [30] M. Gönen and E. Alpaydin, “Localized multiple kernel learning,” in _Machine Learning, Proceedings of the Twenty-Fifth International Conference (ICML 2008), Helsinki, Finland, June 5-9, 2008_ , ser. ACM International Conference Proceeding Series, W. W. Cohen, A. McCallum, and S. T. Roweis, Eds., vol. 307. ACM, 2008, pp. 352–359.
  * [31] J. Yang, Y. Li, Y. Tian, L. Duan, and W. Gao, “Group-sensitive multiple kernel learning for object categorization,” in _2009 IEEE 12th International Conference on Computer Vision_ , 2009, pp. 436–443.
  * [32] Y. Mu and B. Zhou, “Non-uniform multiple kernel learning with cluster-based gating functions,” _Neurocomputing_ , vol. 74, no. 7, pp. 1095–1101, 2011\. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0925231210004571
  * [33] Y. Han and G. Liu, “Probability-confidence-kernel-based localized multiple kernel learning with $l_{p}$ norm,” _IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)_ , vol. 42, no. 3, pp. 827–837, 2012.
  * [34] Y. Han, K. Yang, Y. Yang, and Y. Ma, “Localized multiple kernel learning with dynamical clustering and matrix regularization,” _IEEE Transactions on Neural Networks and Learning Systems_ , vol. 29, no. 2, pp. 486–499, 2018.
  * [35] Y. Han, K. Yang, Y. Yang, and Y. Ma, “On the impact of regularization variation on localized multiple kernel learning,” _IEEE Transactions on Neural Networks and Learning Systems_ , vol. 29, no. 6, pp. 2625–2630, 2018.
  * [36] Y. Lei, A. Binder, U. Dogan, and M. Kloft, “Localized multiple kernel learning—a convex approach,” in _Proceedings of The 8th Asian Conference on Machine Learning_ , ser. Proceedings of Machine Learning Research, R. J. Durrant and K.-E. Kim, Eds., vol. 63. The University of Waikato, Hamilton, New Zealand: PMLR, 16–18 Nov 2016, pp. 81–96. [Online]. Available: https://proceedings.mlr.press/v63/lei63.html
  * [37] C. Hinrichs, V. Singh, J. Peng, and S. Johnson, “Q-mkl: Matrix-induced regularization in multi-kernel learning with applications to neuroimaging,” in _Advances in Neural Information Processing Systems_ , F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, Eds., vol. 25. Curran Associates, Inc., 2012.
  * [38] Y. Han, Y. Yang, X. Li, Q. Liu, and Y. Ma, “Matrix-regularized multiple kernel learning via $(r,~{}p)$ norms,” _IEEE Transactions on Neural Networks and Learning Systems_ , vol. 29, no. 10, pp. 4997–5007, 2018.
  * [39] T. Edmunds, “Protection of 2D face identification systems against spoofing attacks.” Theses, Univ. Grenoble Alpes, Jan. 2017.
  * [40] S. Bhattacharjee, A. Mohammadi, A. Anjos, and S. Marcel, _Recent Advances in Face Presentation Attack Detection_. Cham: Springer International Publishing, 2019, pp. 207–228.
  * [41] R. Ramachandra and C. Busch, “Presentation attack detection methods for face recognition systems: A comprehensive survey,” _ACM Comput. Surv._ , vol. 50, no. 1, pp. 8:1–8:37, Mar. 2017.
  * [42] S. Marcel, M. S. Nixon, J. Fiérrez, and N. W. D. Evans, Eds., _Handbook of Biometric Anti-Spoofing - Presentation Attack Detection, Second Edition_ , ser. Advances in Computer Vision and Pattern Recognition. Springer, 2019.
  * [43] Z. Boulkenafet, J. Komulainen, and A. Hadid, “On the generalization of color texture-based face anti-spoofing,” _Image and Vision Computing_ , vol. 77, pp. 1 – 9, 2018.
  * [44] F. Peng, L. Qin, and M. Long, “Face presentation attack detection using guided scale texture,” _Multimedia Tools and Applications_ , vol. 77, no. 7, pp. 8883–8909, Apr 2018.
  * [45] K. Kollreider, H. Fronthaler, and J. Bigun, “Non-intrusive liveness detection by face images,” _Image and Vision Computing_ , vol. 27, no. 3, pp. 233 – 244, 2009, special Issue on Multimodal Biometrics.
  * [46] L. Feng, L.-M. Po, Y. Li, X. Xu, F. Yuan, T. C.-H. Cheung, and K.-W. Cheung, “Integration of image quality and motion cues for face anti-spoofing: A neural network approach,” _Journal of Visual Communication and Image Representation_ , vol. 38, pp. 451 – 460, 2016.
  * [47] A. Pinto, H. Pedrini, W. R. Schwartz, and A. Rocha, “Face spoofing detection through visual codebooks of spectral temporal cubes,” _IEEE Transactions on Image Processing_ , vol. 24, no. 12, pp. 4726–4740, Dec 2015.
  * [48] J. Galbally and S. Marcel, “Face anti-spoofing based on general image quality assessment,” in _2014 22nd International Conference on Pattern Recognition_ , Aug 2014, pp. 1173–1178.
  * [49] Z. Boulkenafet, J. Komulainen, and A. Hadid, “Face anti-spoofing based on color texture analysis,” in _2015 IEEE International Conference on Image Processing (ICIP)_ , Sept 2015, pp. 2636–2640.
  * [50] S. Jia, X. Li, C. Hu, G. Guo, and Z. Xu, “3d face anti-spoofing with factorized bilinear coding,” _IEEE Transactions on Circuits and Systems for Video Technology_ , vol. 31, no. 10, pp. 4031–4045, 2021.
  * [51] X. Zhu, S. Li, X. Zhang, H. Li, and A. C. Kot, “Detection of spoofing medium contours for face anti-spoofing,” _IEEE Transactions on Circuits and Systems for Video Technology_ , pp. 1–1, 2019.
  * [52] G. Kim, S. Eum, J. K. Suhr, D. I. Kim, K. R. Park, and J. Kim, “Face liveness detection based on texture and frequency analyses,” in _2012 5th IAPR International Conference on Biometrics (ICB)_ , March 2012, pp. 67–72.
  * [53] A. Pinto, W. R. Schwartz, H. Pedrini, and A. d. R. Rocha, “Using visual rhythms for detecting video-based facial spoof attacks,” _IEEE Transactions on Information Forensics and Security_ , vol. 10, no. 5, pp. 1025–1038, May 2015.
  * [54] T. Wang, J. Yang, Z. Lei, S. Liao, and S. Z. Li, “Face liveness detection using 3d structure recovered from a single camera,” in _2013 International Conference on Biometrics (ICB)_ , June 2013, pp. 1–6.
  * [55] H. P. Nguyen, A. Delahaies, F. Retraint, and F. Morain-Nicolier, “Face presentation attack detection based on a statistical model of image noise,” _IEEE Access_ , vol. 7, pp. 175 429–175 442, 2019.
  * [56] G. B. de Souza, J. P. Papa, and A. N. Marana, “On the learning of deep local features for robust face spoofing detection,” _CoRR_ , vol. abs/1806.07492, 2018.
  * [57] J. Yang, Z. Lei, and S. Z. Li, “Learn convolutional neural network for face anti-spoofing,” _CoRR_ , vol. abs/1408.5601, 2014.
  * [58] A. George, Z. Mostaani, D. Geissenbuhler, O. Nikisins, A. Anjos, and S. Marcel, “Biometric face presentation attack detection with multi-channel convolutional neural network,” _IEEE Transactions on Information Forensics and Security_ , vol. 15, pp. 42–55, 2020.
  * [59] B. Hamdan and K. Mokhtar, “The detection of spoofing by 3d mask in a 2d identity recognition system,” _Egyptian Informatics Journal_ , vol. 19, no. 2, pp. 75 – 82, 2018.
  * [60] G. Heusch and S. Marcel, _Remote Blood Pulse Analysis for Face Presentation Attack Detection_. Cham: Springer International Publishing, 2019, pp. 267–289.
  * [61] L. Li, X. Feng, Z. Xia, X. Jiang, and A. Hadid, “Face spoofing detection with local binary pattern network,” _Journal of Visual Communication and Image Representation_ , vol. 54, pp. 182 – 192, 2018.
  * [62] T. Edmunds and A. Caplier, “Motion-based countermeasure against photo and video spoofing attacks in face recognition,” _Journal of Visual Communication and Image Representation_ , vol. 50, pp. 314 – 332, 2018.
  * [63] M. Killioğlu, M. Taşkiran, and N. Kahraman, “Anti-spoofing in face recognition with liveness detection using pupil tracking,” in _2017 IEEE 15th International Symposium on Applied Machine Intelligence and Informatics (SAMI)_ , Jan 2017, pp. 000 087–000 092.
  * [64] J. Yang, Z. Lei, D. Yi, and S. Z. Li, “Person-specific face antispoofing with subject domain adaptation,” _IEEE Transactions on Information Forensics and Security_ , vol. 10, no. 4, pp. 797–809, April 2015.
  * [65] S. R. Arashloo and J. Kittler, “Robust one-class kernel spectral regression,” _IEEE Transactions on Neural Networks and Learning Systems_ , vol. 32, no. 3, pp. 999–1013, 2021.
  * [66] J. von Neumann, “Uber ein okonomisches gleichungssystem und eine verallgemeinerung des brouwerschen fixpunktsatzes,” _Wein 8 Ergebn. Math. Kolloq. Wein_ , pp. 73–83, 1937.
  * [67] D. Du and P. Pardalos, _Minimax and applications_ , ser. Nonconvex Optimization and Its Applications. Kluwer Academic Publishers, 1995.
  * [68] S. P. Boyd and L. Vandenberghe, _Convex Optimization_. Cambridge University Press, 2014.
  * [69] J. Hoffman and S. Frankel, _Numerical Methods for Engineers and Scientists, 2nd ed._ CRC Press, 2001.
  * [70] M. Mohri, A. Rostamizadeh, and A. Talwalkar, _Foundations of Machine Learning_ , 2nd ed., ser. Adaptive Computation and Machine Learning. Cambridge, MA: MIT Press, 2018.
  * [71] G. W. Stewart, _Matrix Algorithms: Volume 1, Basic Decompositions_. Society for Industrial Mathematics, 1998.
  * [72] G. Griffin, A. Holub, and P. Perona, “Caltech256 image dataset,” _Technical Report_ , 2007. [Online]. Available: $http://www.vision.caltech.edu/Image_{D}atasets/Caltech256/$
  * [73] B. Saleh, A. Farhadi, and A. Elgammal, “Object-centric anomaly detection by attribute-based reasoning,” in _2013 IEEE Conference on Computer Vision and Pattern Recognition_ , 2013, pp. 787–794.
  * [74] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” _2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 770–778, 2016.
  * [75] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in _2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2015, pp. 1–9.
  * [76] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in _Advances in Neural Information Processing Systems_ , F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, Eds., vol. 25. Curran Associates, Inc., 2012, pp. 1097–1105.
  * [77] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” _CoRR_ , vol. abs/1409.1556, 2014.
  * [78] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2017, pp. 2261–2269.
  * [79] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” in _2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 4510–4520.
  * [80] B. Zoph, V. Vasudevan, J. Shlens, and Q. V. Le, “Learning transferable architectures for scalable image recognition,” in _2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 8697–8710.
  * [81] P. Perera and V. M. Patel, “Learning deep features for one-class classification,” _CoRR_ , vol. abs/1801.05365, 2018. [Online]. Available: http://arxiv.org/abs/1801.05365
  * [82] M. Everingham, L. V. Gool, C. K. I. Williams, J. Winn, and A. Zisserman, “The pascal visual object classes (voc) challenge,” _International Journal of Computer Vision_ , vol. 88, pp. 303–308, September 2009, printed version publication date: June 2010.
  * [83] R. Chalapathy, A. K. Menon, and S. Chawla, “Anomaly detection using one-class neural networks,” _CoRR_ , vol. abs/1802.06360, 2018. [Online]. Available: http://arxiv.org/abs/1802.06360
  * [84] R. Hadsell, S. Chopra, and Y. LeCun, “Dimensionality reduction by learning an invariant mapping,” in _2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06)_ , vol. 2, 2006, pp. 1735–1742.
  * [85] P. Perera and V. M. Patel, “Learning deep features for one-class classification,” _IEEE Transactions on Image Processing_ , vol. 28, no. 11, pp. 5450–5463, 2019.
  * [86] P. Oza and V. M. Patel, “One-class convolutional neural network,” _IEEE Signal Processing Letters_ , vol. 26, no. 2, pp. 277–281, 2019.
  * [87] A. Costa-Pazo, S. Bhattacharjee, E. Vazquez-Fernandez, and S. Marcel, “The replay-mobile face presentation-attack database,” in _Proceedings of the International Conference on Biometrics Special Interests Group (BioSIG)_ , Sep. 2016.
  * [88] Z. Boulkenafet, J. Komulainen, L. Li, X. Feng, and A. Hadid, “Oulu-npu: A mobile face presentation attack database with real-world variations,” in _2017 12th IEEE International Conference on Automatic Face Gesture Recognition (FG 2017)_ , May 2017, pp. 612–618.
  * [89] D. Wen, H. Han, and A. K. Jain, “Face spoof detection with image distortion analysis,” _IEEE Transactions on Information Forensics and Security_ , vol. 10, no. 4, pp. 746–761, April 2015.
  * [90] I. Chingovska, A. Anjos, and S. Marcel, “On the effectiveness of local binary patterns in face anti-spoofing,” in _2012 BIOSIG - Proceedings of the International Conference of Biometrics Special Interest Group (BIOSIG)_ , Sept 2012, pp. 1–7.
  * [91] “Information technology — biometric presentation attack detection — part 3: Testing and reporting,” International Organization for Standardization, Standard, 2017.
  * [92] W. Sun, Y. Song, C. Chen, J. Huang, and A. C. Kot, “Face spoofing detection based on local ternary label supervision in fully convolutional networks,” _IEEE Transactions on Information Forensics and Security_ , vol. 15, pp. 3181–3196, 2020.
  * [93] Y. Qin, W. Zhang, J. Shi, Z. Wang, and L. Yan, “One-class adaptation face anti-spoofing with loss function search,” _Neurocomputing_ , vol. 417, pp. 384 – 395, 2020.
  * [94] H. Feng, Z. Hong, H. Yue, Y. Chen, K. Wang, J. Han, J. Liu, and E. Ding, “Learning generalized spoof cues for face anti-spoofing.” _CoRR_ , vol. abs/2005.03922, 2020.
  * [95] H. Chen, G. Hu, Z. Lei, Y. Chen, N. M. Robertson, and S. Z. Li, “Attention-based two-stream convolutional networks for face spoofing detection,” _IEEE Transactions on Information Forensics and Security_ , vol. 15, pp. 578–593, 2020.

##  Appendix A Derivation of $\boldsymbol{\mu}$

As noted in Section IV.D, the theorem of minimax can be utilised to change the
minimisation and maximisation order in Eq. (16) to obtain

$\displaystyle\max_{\boldsymbol{\lambda}}-\delta\boldsymbol{\lambda}^{\top}\boldsymbol{\lambda}+2\boldsymbol{\lambda}^{\top}\mathbf{1}+\min_{\boldsymbol{\mu}\in\mathcal{J}}-\boldsymbol{\mu}^{\top}\mathbf{u}$
(26)

where $\mathcal{J}=\\{\boldsymbol{\mu}\Big{|}\boldsymbol{\mu}\geq
0,\left\lVert\boldsymbol{\mu}\boldsymbol{\mu}^{\top}\right\rVert_{p,q}\leq
1\\}$. Let us form the Lagrangian of the minimisation sub-problem in Eq. 26 as

$\displaystyle\mathcal{L}=\gamma(\left\lVert\boldsymbol{\mu}\boldsymbol{\mu}^{\top}\right\rVert_{p,q}-1)-(\mathbf{u}+\boldsymbol{\chi})^{\top}\boldsymbol{\mu}$
(27)

where $\boldsymbol{\chi}\geq 0$ and $\gamma\geq 0$ are non-negative Lagrange
multipliers. The Karush–Kuhn–Tucker (KKT) optimality criteria are:

$\displaystyle\nabla_{\boldsymbol{\mu}}\mathcal{L}=0$ (28a)
$\displaystyle\boldsymbol{\chi}^{\top}\boldsymbol{\mu}=0$ (28b)
$\displaystyle\boldsymbol{\mu}\geq 0$ (28c)
$\displaystyle\gamma(\left\lVert\boldsymbol{\mu}\boldsymbol{\mu}^{\top}\right\rVert_{p,q}-1)=0$
(28d)

Using (28a) and (28c) one obtains

$\displaystyle-\mathbf{u}-\boldsymbol{\chi}+\gamma\Big{[}\frac{\left\lVert\boldsymbol{\mu}\right\rVert_{p}\left\lVert\boldsymbol{\mu}\right\rVert_{q}}{\left\lVert\boldsymbol{\mu}\right\rVert_{p}^{p}}\boldsymbol{\mu}^{p-1}+\frac{\left\lVert\boldsymbol{\mu}\right\rVert_{q}\left\lVert\boldsymbol{\mu}\right\rVert_{p}}{\left\lVert\boldsymbol{\mu}\right\rVert_{q}^{q}}\boldsymbol{\mu}^{q-1}\Big{]}=0$
(29)

By inspecting the minimisation subproblem in Eq. 26, one understands that the
elements of $\boldsymbol{\mu}$ should be maximised for optimality. As the
matrix norm is a convex function of $\boldsymbol{\mu}$, maximisation of the
components of $\boldsymbol{\mu}$ maximises
$\left\lVert\boldsymbol{\mu}\boldsymbol{\mu}^{\top}\right\rVert_{p,q}$ whose
maximum occurs on the border of the set $\mathcal{J}$ which is given as
$\left\lVert\boldsymbol{\mu}\boldsymbol{\mu}^{\top}\right\rVert_{p,q}=1$. As
we have
$\left\lVert\boldsymbol{\mu}\boldsymbol{\mu}^{\top}\right\rVert_{p,q}=\left\lVert\boldsymbol{\mu}\right\rVert_{p}\left\lVert\boldsymbol{\mu}\right\rVert_{q}$
[38], at the optimal point it holds
$\left\lVert\boldsymbol{\mu}\right\rVert_{p}\left\lVert\boldsymbol{\mu}\right\rVert_{q}=1$.
Eq. 29 can now be expressed as

$\displaystyle\mathbf{u}+\boldsymbol{\chi}=\gamma\Big{(}\frac{\boldsymbol{\mu}^{p-1}}{\left\lVert\boldsymbol{\mu}\right\rVert_{p}^{p}}+\frac{\boldsymbol{\mu}^{q-1}}{\left\lVert\boldsymbol{\mu}\right\rVert_{q}^{q}}\Big{)}=\gamma\boldsymbol{\mu}\odot\Big{(}\frac{\boldsymbol{\mu}^{p-2}}{\left\lVert\boldsymbol{\mu}\right\rVert_{p}^{p}}+\frac{\boldsymbol{\mu}^{q-2}}{\left\lVert\boldsymbol{\mu}\right\rVert_{q}^{q}}\Big{)}$
(30)

Eq. 30 may be rewritten as

$\displaystyle\bar{\mathbf{u}}+\bar{\boldsymbol{\chi}}=\gamma\boldsymbol{\mu}$
(31)

where $\bar{\mathbf{u}}$ and $\bar{\boldsymbol{\chi}}$ are defined as

$\displaystyle\bar{\mathbf{u}}=\mathbf{u}\odot\Big{(}\frac{\boldsymbol{\mu}^{p-2}}{\left\lVert\boldsymbol{\mu}\right\rVert_{p}^{p}}+\frac{\boldsymbol{\mu}^{q-2}}{\left\lVert\boldsymbol{\mu}\right\rVert_{q}^{q}}\Big{)}^{-1}$
(32)

and

$\displaystyle\bar{\boldsymbol{\chi}}=\boldsymbol{\chi}\odot\Big{(}\frac{\boldsymbol{\mu}^{p-2}}{\left\lVert\boldsymbol{\mu}\right\rVert_{p}^{p}}+\frac{\boldsymbol{\mu}^{q-2}}{\left\lVert\boldsymbol{\mu}\right\rVert_{q}^{q}}\Big{)}^{-1}$
(33)

As $\boldsymbol{\mu}\geq 0$, $\mathbf{u}\geq 0$ and $\boldsymbol{\chi}\geq 0$,
one has $\bar{\mathbf{u}}\geq 0$, $\bar{\boldsymbol{\chi}}\geq 0$. From Eq. 31
we have

$\displaystyle\boldsymbol{\mu}=(\bar{\mathbf{u}}+\bar{\boldsymbol{\chi}})/\gamma$
(34)

Due to (28b) it must hold:

$\displaystyle\boldsymbol{\chi}^{\top}\boldsymbol{\mu}=\frac{1}{\gamma}\sum_{c,g}\chi_{c,g}(\bar{u}_{c,g}+\bar{\chi}_{c,g})=0$
(35)

Note that in order for Eq. 35 to be satisfied, it should hold that
$\boldsymbol{\chi}=0$. In order to illustrate this, we employ contradiction
and suppose all components of $\boldsymbol{\chi}$ are not zero and there
exists an arbitrary index $j$ for which $\chi_{j}=\epsilon>0$. Such an
assumption results in

$\displaystyle\boldsymbol{\chi}^{\top}\boldsymbol{\mu}=\frac{\epsilon}{\gamma}\Big{(}\frac{\mu_{j}^{p-2}}{\left\lVert\boldsymbol{\mu}\right\rVert_{p}^{p}}+\frac{\mu_{j}^{q-2}}{\left\lVert\boldsymbol{\mu}\right\rVert_{q}^{q}}\Big{)}^{-1}\Big{(}{u}_{j}+\epsilon\Big{)}>0$
(36)

which opposes the condition $\boldsymbol{\chi}^{\top}\boldsymbol{\mu}=0$.
Hence, $\boldsymbol{\chi}$ does not include any non-zero components, and as a
result, $\boldsymbol{\chi}=0$ which yields $\bar{\boldsymbol{\chi}}=0$. Based
on Eq. 34, $\boldsymbol{\mu}$ is determined as

$\displaystyle\boldsymbol{\mu}=\mathbf{\bar{u}}/\gamma$ (37)

and due to
$\left\lVert\boldsymbol{\mu}\right\rVert_{p}\left\lVert\boldsymbol{\mu}\right\rVert_{q}=1$,
one obtains

$\gamma=\sqrt{\left\lVert\bar{\mathbf{u}}\right\rVert_{p}\left\lVert\bar{\mathbf{u}}\right\rVert_{q}}$
(38)

##  Appendix B Proof of Proposition 1

As discussed in Section IV.D, thanks to the introduction of vector
$\boldsymbol{\mu}$ as the collection of kernel weights from all clusters, the
optimisation problem associated with the proposed approach given in Eq. (17)
exactly matches that of [8] given in Eq. (6). Consequently, following a
similar optimisation approach, the convergence guarantees presented in [8]
directly carry over to the proposed localised approach. More specifically, for
a sufficiently large $\delta$, Algorithm 1 is convergent to a single fixed-
point independent of the initialisation of $\boldsymbol{\lambda}$.

##  Appendix C Proof of Proposition 2

In order to compute $\mathcal{R}(H^{p,q}_{CG})$, let us first consider the
following theorem from [70]

######  Theorem 1 (Rademacher complexity of a kernel-based hypothesis)

Assume $\kappa:\mathcal{X}\times\mathcal{X}\rightarrow\mathbb{R}$ is a
positive definite symmetric kernel function and suppose
$\psi:\mathcal{X}\rightarrow\mathbb{H}$ is a feature projection linked to
$\kappa$. If $S\subseteq\\{x:\kappa(x,x)\leq r^{2}\\}$ is a dataset of size
$n$, and
$\mathbb{H}=\\{x\rightarrow\langle\mathbf{w},\psi(x)\rangle:\left\lVert\mathbf{w}\right\rVert_{\mathbb{H}}\leq\Lambda\\}$
for some $\Lambda\geq 0$ denotes the hypothesis class, then, the empirical
Rademacher complexity of $\mathbb{H}$ over $S$ is bounded as

$\displaystyle\mathcal{R}(\mathbb{H})\leq\frac{\Lambda\sqrt{\mathrm{Tr}[\mathbf{K}]}}{n}$
(39)

where $\mathrm{Tr}[\mathbf{K}]$ denotes the trace of the kernel matrix
$\mathbf{K}$.

Note that in the proposed approach, we have
$\mathbf{K}=\sum_{c}\sum_{g}\mu_{cg}\mathbf{K}_{cg}$, and hence

$\displaystyle\mathrm{Tr}[\mathbf{K}]$ $\displaystyle=$
$\displaystyle\sum_{i}\sum_{c}\sum_{g}\mu_{cg}p^{2}_{c}(\mathbf{x}_{i})\kappa_{g}(\mathbf{x}_{i},\mathbf{x}_{i})$
(40) $\displaystyle\leq$ $\displaystyle
r^{2}\sum_{c}\sum_{g}\mu_{cg}\sum_{i}p^{2}_{c}(\mathbf{x}_{i})$
$\displaystyle\leq$
$\displaystyle\Bigg{(}r^{2}\sum_{c}\sum_{g}\mu_{cg}\Bigg{)}\Bigg{(}\sum_{c}\sum_{i}p^{2}_{c}(\mathbf{x}_{i})\Bigg{)}$

where the second inequality is correct as $\mu_{cg}$’s and
$p_{c}(\mathbf{x}_{i})$’s are non-negative. Note that
$\sum_{c}\sum_{g}\mu_{cg}=\left\lVert\boldsymbol{\mu}\right\rVert_{1}$. Using
an $\ell_{1}$\- to an $\ell_{p}$-norm ($p\geq 1$) conversion [29], we have

$\displaystyle\left\lVert\boldsymbol{\mu}\right\rVert_{1}\leq(CG)^{1-1/p}\left\lVert\boldsymbol{\mu}\right\rVert_{p}$
(41)

Similarly, for $q\geq 1$ we have

$\displaystyle\left\lVert\boldsymbol{\mu}\right\rVert_{1}\leq(CG)^{1-1/q}\left\lVert\boldsymbol{\mu}\right\rVert_{q}$
(42)

Multiplying the two inequalities above, one obtains

$\displaystyle\left\lVert\boldsymbol{\mu}\right\rVert_{1}^{2}$
$\displaystyle\leq$
$\displaystyle(CG)^{2-1/p-1/q}\left\lVert\boldsymbol{\mu}\right\rVert_{p}\left\lVert\boldsymbol{\mu}\right\rVert_{q}$
(43) $\displaystyle=$
$\displaystyle(CG)^{2-1/p-1/q}\left\lVert\boldsymbol{\mu}\boldsymbol{\mu}^{\top}\right\rVert_{p,q}$

Since in the proposed approach we have
$\left\lVert\boldsymbol{\mu}\boldsymbol{\mu}^{\top}\right\rVert_{p,q}\leq 1$,
one obtains

$\displaystyle\sum_{c}\sum_{g}\mu_{cg}=\left\lVert\boldsymbol{\mu}\right\rVert_{1}\leq(CG)^{1-\frac{1}{2p}-\frac{1}{2q}}$
(44)

Consequently, using Theorem 1 and Eq. 40, the empirical Rademacher complexity
of the proposed approach is bounded as

$\displaystyle\mathcal{R}(H_{CG}^{p,q})\leq\frac{\Lambda
r}{n}(CG)^{\frac{1}{2}-\frac{1}{4p}-\frac{1}{4q}}\Bigg{(}\sum_{c}\sum_{i}p_{c}^{2}(\mathbf{x}_{i})\Bigg{)}^{1/2}$
(45)

##  Appendix D Dependence of $\Lambda$ on $\delta$

Since
$\boldsymbol{\lambda}^{\top}\mathbf{K}\boldsymbol{\lambda}\leq\Lambda^{2}$,
and as the kernel matrix is constant given that the kernel function and the
data are chosen and fixed, the upper bound on
$\boldsymbol{\lambda}^{\top}\mathbf{K}\boldsymbol{\lambda}$ may be minimised
by reducing the magnitude of $\boldsymbol{\lambda}$ which is controlled as

$\displaystyle\left\lVert\boldsymbol{\lambda}\right\rVert_{2}$
$\displaystyle=$
$\displaystyle\left\lVert\Big{(}\delta\mathbf{I}+\mathbf{K}\Big{)}^{-1}\mathbf{1}\right\rVert_{2}\leq\left\lVert\Big{(}\delta\mathbf{I}+\mathbf{K}\Big{)}^{-1}\right\rVert_{2}\left\lVert\mathbf{1}\right\rVert_{2}$
(46) $\displaystyle=$
$\displaystyle\frac{\sqrt{n}}{\vartheta+\delta}\leq\frac{\sqrt{n}}{\delta}=\frac{\theta}{\sqrt{n}}$

where the first equality is true since the norm of a vector-matrix product is
less than or equal to their norms multiplied. In the equation above,
$\vartheta$ stands for the least significant eigenvalue of the combined kernel
matrix. Note that the eigenvalues are shifted by the amount of the constant
added to the main diagonal and the norm of an inverse matrix is equal to the
reciprocal of the minimal eigenvalue of the primary matrix. According to Eq.
46, by increasing $\delta$, the magnitude of $\boldsymbol{\lambda}$ reduces,
and consequently, the upper bound on
$\boldsymbol{\lambda}^{\top}\mathbf{K}\boldsymbol{\lambda}$ may be reduced and
subsequently, the empirical Rademacher complexity is decreased. However,
shrinking the elements of $\boldsymbol{\lambda}$ may lead to an increase in
the training error loss. As such, the optimal $\delta$ (or equivalently
$\theta$) may be determined via cross validation.

##  Appendix E Proof of Proposition 4

By inspecting Eq. 40, it is clear that different regularisation constraints
shall only lead to differences on the bounds obtained for the
”$\sum_{c}\sum_{g}\mu_{cg}$” term.

  1. (i)

For the joint vector $\ell_{p}$-norm regularisation
($\left\lVert\boldsymbol{\mu}\right\rVert_{p}\leq 1$), using an
$\ell_{1}$-to-$\ell_{p}$ conversion we have

$\displaystyle\sum_{c}\sum_{g}\mu_{cg}=$
$\displaystyle\left\lVert\boldsymbol{\mu}\right\rVert_{1}\leq(CG)^{1-1/p}\left\lVert\boldsymbol{\mu}\right\rVert_{p}\mathrm{}$
(47) $\displaystyle\overset{\left\lVert\boldsymbol{\mu}\right\rVert_{p}\leq
1}{\leq}(CG)^{1-1/p}$

  2. (ii)

For the disjoint vector $\ell_{p}$-norm regularisation
($\left\lVert\boldsymbol{\mu}_{c}\right\rVert_{p}\leq 1$, $c=1,\dots,C$) we
have

$\displaystyle\sum_{c}\sum_{g}\mu_{cg}$ $\displaystyle=$
$\displaystyle\sum_{c}\left\lVert\boldsymbol{\mu}_{c}\right\rVert_{1}\leq\sum_{c}G^{1-1/p}\left\lVert\boldsymbol{\mu}_{c}\right\rVert_{p}$
(48)
$\displaystyle\overset{\left\lVert\boldsymbol{\mu}_{c}\right\rVert_{p}\leq
1}{\leq}CG^{1-1/p}$

  3. (iii)

And, for the disjoint matrix-norm regularisation
($\left\lVert\boldsymbol{\mu}_{c}\boldsymbol{\mu}_{c}^{\top}\right\rVert_{p,q}\leq
1$, $c=1,\dots,C$), using $\ell_{1}$-to-$\ell_{p}$ and
$\ell_{1}$-to-$\ell_{q}$ conversions for $p,q\geq 1$, we have

$\displaystyle\left\lVert\boldsymbol{\mu}_{c}\right\rVert_{1}\leq
G^{1-1/p}\left\lVert\boldsymbol{\mu}_{c}\right\rVert_{p}$
$\displaystyle\left\lVert\boldsymbol{\mu}_{c}\right\rVert_{1}\leq
G^{1-1/q}\left\lVert\boldsymbol{\mu}_{c}\right\rVert_{q}$ (49)

Multiplying the two inequalities above, one obtains

$\displaystyle\left\lVert\boldsymbol{\mu}_{c}\right\rVert_{1}^{2}$
$\displaystyle\leq$ $\displaystyle
G^{2-\frac{1}{p}-\frac{1}{q}}\left\lVert\boldsymbol{\mu}_{c}\right\rVert_{p}\left\lVert\boldsymbol{\mu}_{c}\right\rVert_{q}$
(50) $\displaystyle=$ $\displaystyle
G^{2-\frac{1}{p}-\frac{1}{q}}\left\lVert\boldsymbol{\mu}_{c}\boldsymbol{\mu}_{c}^{\top}\right\rVert_{p,q}$

Since
$\left\lVert\boldsymbol{\mu}_{c}\boldsymbol{\mu}_{c}^{\top}\right\rVert_{p,q}\leq
1$, $\left\lVert\boldsymbol{\mu}_{c}\right\rVert_{1}$ is bounded as

$\displaystyle\left\lVert\boldsymbol{\mu}_{c}\right\rVert_{1}\leq
G^{1-\frac{1}{2p}-\frac{1}{2q}}$ (51)

Consequently, for the disjoint matrix-norm class we have

$\displaystyle\sum_{c}\sum_{g}\mu_{cg}$ $\displaystyle=$
$\displaystyle\sum_{c}\left\lVert\boldsymbol{\mu}_{c}\right\rVert_{1}\leq\sum_{c}G^{1-\frac{1}{2p}-\frac{1}{2q}}$
(52) $\displaystyle\leq$ $\displaystyle CG^{1-\frac{1}{2p}-\frac{1}{2q}}$

Comparing the bounds in Eq.’s 47, 48, and 52 with that of the proposed joint
matrix-norm approach given in Eq. 44, the proposition is derived.$\square$