# Few-Shot Learning for Biometric Verification

Saad Bin Ahmed<EMAIL_ADDRESS>
Department of Computer Science
Middlesex College, Western University
London, Ontario N6A 5B7, Canada Umaid M. Zaffar<EMAIL_ADDRESS>
Department of Data Science
University of Waterloo, 200 University Avenue West
Waterloo, ON, Canada N2L 3G1 Marium Aslam<EMAIL_ADDRESS>
School of Electrical Engineering and Computer Science
National University of Science and Technology
Islamabad, Capital 44000, Pakistan Muhammad Imran Malik
<EMAIL_ADDRESS>
School of Electrical Engineering and Computer Science
National University of Science and Technology
Islamabad, Capital 44000, Pakistan

###### Abstract

In machine learning applications, it is common practice to feed as much
information as possible to the model to increase accuracy. However, in cases
where training data is scarce, Few-Shot Learning (FSL) approaches aim to build
accurate algorithms with limited data. In this paper, we propose a novel
lightweight end-to-end architecture for biometric verification that produces
competitive results compared to state-of-the-art accuracies through FSL
methods. Unlike state-of-the-art deep learning models with dense layers, our
shallow network is coupled with a conventional machine learning technique that
uses hand-crafted features to verify biometric images from multi-modal
sources, including signatures, periocular regions, iris, face, and
fingerprints. We introduce a self-estimated threshold that strictly monitors
False Acceptance Rate (FAR) while generalizing its results to eliminate user-
defined thresholds from ROC curves that may be biased by local data
distribution. Our hybrid model benefits from few-shot learning to compensate
for the scarcity of data in biometric use-cases. We conducted extensive
experiments on commonly used biometric datasets, and our results demonstrate
the effectiveness of our proposed solution for biometric verification systems.

Keywords: Verification, Few-Shot Learning, One-Class Support Vector Machines

##  1 Introduction

Deep Learning has gained popularity in the past decade, enabling the
development of methods for visual recognition problems (Krizhevsky et al.,
2012; Patil and Banyal, 2019; Shubathra et al., 2020). These methods extract
features and learn to associate weights with each feature using back-
propagation algorithms. Furthermore, such models are commonly used in
verification systems that authenticate users for device and network access.
For example, handwriting style and facial expressions can be used for user
authentication, as depicted in Figure 1.

Biometric verification requires a model that can filter identifying
information pertaining to an individual, which necessitates a predetermined
threshold indicating the minimum score required to classify a test image as a
genuine sample. Receiver Operating Characteristic (ROC) curves (Bradley, 1997)
are commonly used to determine the threshold value where the accuracy of the
classification model is highest and the sum of True Positives and True
Negatives is maximum. However, biometric verification requires maximum
specificity in terms of features to prevent spoofing. The ROC curves are
predetermined and computed by iteratively testing the model at different
threshold values. In an imbalanced dataset, classification bias can be caused
by certain features dominating the samples (Krishnapriya et al., 2020). This
bias is propagated in the ROC curve, leading to unreliable algorithm
performance. In this paper, we propose to replace this mechanism with a self-
computed threshold for each input object customized according to its intra-
class variation, called the confidence factor. This indicates the confidence
that one classifier has relative to another classifier while minimizing the
False Acceptance Rate (FAR).

Figure 1: Biometric image samples from UBIPr, SigComp’11 and SigWiComp’13

While machine learning algorithms have proven efficient in data-intensive
applications, they often struggle when input data is scarce, which poses a
particular challenge for biometric verification systems that require high
accuracy to detect anomalies.

Meta-learning is a subset of machine learning that teaches itself to learn
from transferable knowledge (Finn et al., 2017), and it can be fine-tuned to
custom data. One type of meta-learning that has gained prominence in recent
years is Few-Shot Learning, which involves predicting and classifying new data
with limited training samples in a supervised learning setting. Few-shot
learning is a rapidly advancing field that enables the use of sparse data to
achieve successful results (Wang et al., 2020). In this study, we propose an
architecture for configuring biometric verification systems that allows for
the verification of multiple biometric datasets from different sources in a
way that yields competitive results.

In this paper, we aim to make the following contributions.

  1. 1.

We propose a few-shot learning architecture that maintains memory and speed
efficiency while achieving fast and reliable results on multiple biometric
modalities. This architecture couples a neural network with a boundary-based
classifier.

  2. 2.

We introduce a novel method for calculating the verification threshold, which
involves computing the distance from the decision boundary and applying an
inverse sigmoid mapping. This method minimizes the False Acceptance Rate (FAR)
and achieves state-of-the-art accuracy.

  3. 3.

To compensate for the lack of transparency in neural networks, we use hand-
crafted features in conjunction with deep features. This hybrid approach
produces efficient and competitive results.

##  2 Related Work

While few-shot deep learning methods for classification have seen substantial
progress, data-efficient approaches to deep anomaly detection are still a
critical research direction. These anomaly detection schemes, often involving
semi-supervised learning, are equipped with mechanisms that detect outliers
from a normal class. One-Class Support Vector Machine (OC-SVM) (Schölkopf et
al., 1999) maps input data onto a plane and draws a decision boundary, called
the hyperplane, to separate genuine and anomalous classes. Support Vector Data
Description (SVDD) (Tax and Duin, 2004) is related to OC-SVM with the
objective to find the smallest hypersphere that envelops the data such that
radius $R>0$ and center C belongs to the feature space of its associated
feature mapping. Similarly, Auto encoders were first used for outlier
detection by (Hawkins et al., 2002) by reconstructing data to create cluster
groups, enabling the detection of anomalies. In recent studies, numerous
verification-based models have been proposed that conduct verification using
deep neural networks (Blanchard et al., 2010; Perera and Patel, 2019). These
solutions are performed under different paradigms such as deep hybrid learning
(Erfani et al., 2016; Wu et al., 2015), supervised learning (Erfani et al.,
2017; Görnitz et al., 2013) and semi-supervised learning (Blanchard et al.,
2010; Perera and Patel, 2019).

Deep Hybrid Learning uses deep learning to extract a set of representative and
robust features from unstructured data that are fed into traditional kernel-
based anomaly detection techniques, employing a sequential process of
extracting features using a neural network and then using machine learning
approaches to create a classification model (Hodge and Austin, 2018). In this
paper, we present a hybrid model that does not rely on this sequential
process. Our deep learning model and traditional kernel-based method are, in
fact, independent of one another and merged together by an inverse sigmoid
mapping of the distance of data points from the decision boundary that serves
to estimate the confidence factor for each input data.

Traditionally, hand-crafted features such as Local Binary Pattern (Ojala et
al., 2002), Speeded Up Robust Features (Zhu et al., 2018) and Local Phase
Quantization (Ojansivu and Heikkilä, 2008)have been used for classification.
These features provide users with transparency in decision-making, allowing
them to fine-tune their models. However, biometric verification systems
require the detection of imposters who can emulate visible features, making it
essential to use deep models that can extract representative features. While
these models perform well in data-intensive situations, they struggle with
small datasets. Some biometric verification algorithms, such as the two-stage
Error Weighted Fusion algorithm (Uzair et al., 2015), require significant
manual intervention to remove disoriented data, which can be impractical.
Therefore, deep learning methods that are capable of handling sparse datasets
are essential for effective biometric verification.

Figure 2: The relation network architecture by Sung et al. (2018). The
concatenation layer combines the embedding from the query set and the sample
set as both undergo the same process for the first four layers. Following the
concatenation layer, the combined feature embedding is forwarded to the next
layers.

The availability of data is a significant issue in biometric verification
applications. Recently, many Few-Shot Learning (FSL) models have been proposed
for evaluating images under sparse data (Fei-Fei et al., 2006; Lake et al.,
2011). FSL techniques aim to learn a target class T with limited, supervised
data by generalizing the model using prior knowledge for task-specific data
that follows the same distribution as trained data (Wang et al., 2020). There
are four types of techniques based on prior knowledge:

  1. 1.

Multi-task learning: These algorithms learn multiple related tasks
simultaneously by exploiting both task-specific and task-generic methods,
making them naturally suitable for FSL (Caruana, 1998).

  2. 2.

Embedding learning: TThis method embeds samples from training data into a
lower dimension that is less complex, allowing similar data to cluster
together while distinguishing dissimilar data conveniently (Jia et al., 2014;
Wang et al., 2020).

  3. 3.

Learning with External Memory: In this method, data from the training set is
stored in an external memory in key-value format, and each test sample
represents the contents of the memory as a weighted average (Miller et al.,
2016; Sukhbaatar et al., 2015).

  4. 4.

Generative Modeling: This method estimates the probability distribution $p(x)$
from the observed $x_{i}$’s, where i is the index of observed data, using
prior knowledge and involving the estimation of $p(y|x)$ and $p(y)$ (Rezende
et al., 2016).

The proposed model builds upon the work of Sung et al. (2018), which uses
embedding learning to train a two-branched Relation Network (RN) that compares
query images to few-shot labelled sample images. In this paper, we introduce a
modified version of the RN architecture for biometric verification. Our model
compares query images to a set of K-shot samples, using only genuine images in
the sample set. To minimize biases within classes, as explained by Sung et al.
(2018) we inhibit the element-wise summation of embedding modules, as depicted
in Figure 2. We derive the similarity score threshold using a confidence
factor extracted from kernel-based estimation. Our simple approach produces
competitive results and offers significant improvements in FAR compared to
previous approaches, providing a single model solution to multi-modal
biometric sources.

Figure 3: The relation network architecture used in proposed biometric
verification system. The sample set consists of images that are genuine and
the query set image is being questioned for its authenticity. After comparing
the query image with sample sets, the embedding that yields highest score is
the most similar image through deep feature comparison.

##  3 Problem Definition

For the purpose of biometric verification, we are examining the task of few-
shot learning. In this section, the proposed system is divided into two parts:
a few-shot learning model and a threshold computing mechanism for
authentication. We employ few-shot learning algorithm that uses Relation
Networks which are also required to have a training set, support set and a
test set. The support set and test set have the same label space and the
support set contains $K$ labelled examples for each of $C$ unique classes, ,
calling the target few-shot problem a $C$-way-$K$-shot problem.

In principle, the support set can be used to train the classifier but because
of the limited label space, the performance is, usually, not satisfactory.
Therefore, we perform meta-learning on training set, which has its own label
space disjoint from the support set, in order to extract transferable
knowledge. The training set consists of $N_{i}^{g}$ number of images from each
class $i$ from the genuine images $g$. In each class, there are genuine images
that are anomalous for other classes. In the meta-learning phase, this set is
used to train the relation network that learns to compare a pair of two
images. We emulate the episodic training setting proposed by Vinyals et al.
(2016) to benefit from our complete training set where, in every episode
random set of sample and query images are used to maximize generalization and
test accuracy. The sample set $S$ contains $K$ reference images from each of
the selected $C$ classes, from the $G$ genuine classes, $S=\\{(x)\\}^{m}$
where $m=K\times C$ and $x$ represents a tuple of sample images and labels.
The query set contains $N$ number of images from the same $C$ classes,
$Q=\\{(y)\\}^{n}$ where $n=N\times C$ and y represents the tuple of query
images and label. All $n\times m$ images are encoded by the feature encoder
and produce $n\times m$ embeddings. Each query embedding, $f(y_{i})$, is
batched with each of the $C$ sample embeddings creating $n\times C$ pairs per
episode. Considering a single batch, with respect to a query embedding, we
would have $C$ pairs of embeddings in which the embedding $f(y_{i})$ is
concatenated with $C$ sample embeddings. Out of these $C$ pairs, only one is
the correct genuine pair, while the others are anomalous. We opt for a
modified version of 5-way-1-shot learning setting as illustrated in figure 3
for our model training to provide for the possibility of forgeries in our
dataset. For threshold computation, we devise an independent data-
preprocessing pipeline to extract hand-crafted features and utilize them to
perform one-class classification using the One-Class SVM on the test feature
vector against the target class. The distance from the hyperplane to the test
sample is further processed using a scaling function and exploited via an
Inverse Sigmoid Mapping to extract the confidence factor based on the
distance. This factor is used as a threshold value for the few-shot RN
relation score. The detailed pipeline is described in Section 4.

##  4 Model

The biometric verification architecture is constructed around two networks, as
illustrated in Figure 6: OC-SVM that extracts a confidence factor using hand-
crafted features and another Relation Network that learns to compare images
based on their deep embeddings. Each task is represented by an image from a
unimodal source that is sub-divided into genuine and imposter set that
constitutes $N^{g}$ and $N^{f}$ images. After a series of preprocessing steps
extracting hand-crafted feature descriptors from an image $g_{i}$ is fed to
OC-SVM. For each $g_{i}$ that is mapped on the plane, the distance $d_{i}$
from the hyperplane is measured which is input to a sigmoid function
$\sigma(d_{i})$. We subtract one from this value to calculate the inverse
mapping $O_{i}$ which is used to compute the confidence factor,

$O_{i}=1-\sigma(d_{i}),i=1,2,3,...,N^{g}$ (1)

A numerical value is assigned to the OC-SVM decision based on this output.
Using this threshold value as a level against the relational score from RN
allows us to maximize our accuracy while minimizing FAR. A detailed overview
of all modules involved in this process is as follows,

Figure 4: The data preprocessing pipeline is used for all modes of biometric
datasets. These set of techniques allows us to quantize the hand-picked
feature so that we can produce our OC-SVM model.

###  4.1 Preprocessing

$N^{g}$ images compose the target class while $N^{f}$ images constitute the
rest of the classes augmented as an anomalous class. We use OC-SVM, a class
boundary based classification technique, to train our kernel based model.
Therefore, only $N^{g}$ images needs to be preprocessed. To efficiently
extract features from the images, Oriented FAST and Rotated Brief (Rublee et
al., 2011) are applied. For each image $g_{i}$, the number of local feature
key points vary under different environmental conditions such as light,
orientation, angle and shadow. In order to resolve this inconsistency, we
employ a bag of visual words to constrain the number of feature descriptors
for each image to a set number of visual words. This is achieved by using a
K-Means clustering algorithm trained on each raw feature descriptor of every
image $g_{i}$. $N^{g}$ images are fed to K-Means Cluster and a normalized
histogram is created that represents the occurrences of key points in each
cluster of an image. We store this preprocessed data for each image that
represents a bag-of-visual-words. This technique runs a risk of having empty
clusters that can spoil the dataset. Therefore, a Principle Component Analysis
(PCA) followed by this technique with an intention to reduce the dimensions of
feature descriptor. The sequential steps are outlined in Figure 4.

###  4.2 One-Class SVM

The previously determined feature vectors for our genuine set is input to OC-
SVM Schölkopf et al. (1999) to develop a model that separates the genuine
images from the origin so that the distance between the hyperplane and origin
is maximized. In our work, the distance between the decision boundary and the
data points plays a significant role as a confidence inducer. Using this
model, a sample image can either be classified as genuine or anomalous based
on its position relative to the decision boundary. In this initial estimation,
positive distances and negative distances are calculated to represent genuine
and anomalous class respectively. The sigmoid function is used to infer the
confidence factor for the test image as represented in equation (1). The
distance $d_{i}$ is mapped to a range of $[0,1]$ as an output of sigmoid
function. Ideally, according to this function, the test samples classified as
anomalous are mapped to $[0,0.5]$ and genuine samples to $[0.5,1]$.

The probability of misclassification increases if the test sample lies close
to the decision boundary depending on intra-class variation. Since these
samples have a similar risk of misclassification, their inverse sigmoid output
$O_{i}$ is estimated close to 0.5. This confidence factor will give an equal
chance to the relation network to compute the relational score that will
determine the final verdict regarding the distribution of sample data. The
inverse sigmoid mapping is explained under four scenarios that correspond to
OC-SVM and how any data that is initially mislabelled can be amended. An ideal
scenario is illustrated in Figure 5 where such data points are processed using
OC-SVM to show the concept of using the distance to compute the confidence
factor.

Figure 5: Data points as they are mapped on a 2-D plane and classified
according to their distance from the decision boundary which is then mapped
using inverse sigmoid function that gives us thresholds as shown

####  4.2.1 True Positives (TP)

Test samples that are evidently similar to the genuine set based on visual
observation and handcrafted features are likely to be mapped furthest from the
hyperplane towards the positive direction, yielding a confidence factor of
$0.5<<\sigma(d_{i})<1$. This indicates that the model has inferred a high
probability for the sample test, thereby placing confidence in its result as a
genuine class. When the sigmoid function is inverted, it will result in an
output of $0<O_{i}<<0.5$. , which represents a low threshold for the
relational score. This is because the sample is already evidently similar to
the genuine distribution, and does not require thorough confirmation via deep
feature comparison to classify it otherwise. However, due to high intra-class
variation, the sample may lie near the decision boundary of the OC-SVM. In
this scenario, the $O_{i}$ will yield an output close to 0.5, and the relation
network will play its role in conducting a deep feature comparison of
embeddings that will yield a result greater than this threshold.

####  4.2.2 True Negatives (TN)

If the test samples are correctly classified by the OC-SVM as anomalous class,
then the distance from the decision boundary will be in the negative
direction. If the sample image is evidently dissimilar to the genuine set, the
sigmoid function will give $0<\sigma(d_{i})<<0.5$, and likewise,
$0.5<<O_{i}<1$. This implies that the sample image will require a high
relational score based on deep feature comparison to overcome its
classification as an anomalous class. As in the case of TP, the sample image
can also lie near the decision boundary if there is high intra-class variation
in the genuine set. In that case, the relation network will output a
relational score less than $O_{i}$ to confirm that the sample image is
anomalous.

####  4.2.3 False Positives (FP)

The test samples that exhibit some similarity to the images in the genuine
set, but do not correspond to its distribution, will lie near the decision
boundary as they are neither completely similar nor dissimilar to the genuine
set. In this case, the sigmoid function will output $0.5<\sigma(d_{i})<<1$,
and corresponding, $0<<O_{i}<0.5$. Therefore, the relation network will be
responsible for comparing embeddings to give a relational score less than the
confidence factor, and correct the mislabeled sample image.

####  4.2.4 False Negatives (FN)

The test samples are slightly dissimilar to the distribution of the genuine
set; however, they do belong to the genuine class. For example, a tired
person’s signature may be slightly different compared to when the person was
alert. This sample also lies near the decision boundary towards the negative
direction with $0<<\sigma(d_{i})<0.5$ and $0.5<O_{i}<<1.$ TThe relation
network will output a relational score greater than $O_{i}$,, correcting the
earlier label of this sample.

###  4.3 Relation Network

The Relation Network constitutes the second module of our architecture, which
is responsible for conducting few-shot learning and relation score estimation
for biometric verification, as previously mentioned. The CNN architecture
comprises two sub-networks: the Feature Encoding Module and the Relation
Module. The Feature Encoder generates embeddings of each image independently,
and the resultant class sample and query sample embeddings are concatenated to
form sample-query pairs. These pairs are fed into the Relation Module, which
outputs the relation score of each pair of embeddings. We can define each
episodic batch as a set that consists of one genuine sample image of the
target class, four random anomalous class samples, and a query image belonging
to the target class. The network not only learns to discover similarities
between genuine class samples but also to determine differences between the
query sample and anomalous classes. The output relation scores, ranging from 0
and 1, describe how similar the query image is to the genuine class sample and
how different it is from the anomalous class samples. Mean Squared Error (MSE)
is calculated, where the genuine score is compared with 1 and the anomalous
scores with 0. This error is back-propagated as the loss function of the
network, updating the corresponding weights of the Feature Encoder and the
Relation Module sub-networks. In multi-class datasets with no forgery data,
the training is conducted similarly to Sung et al. (2018), where the network
learns to compare and differentiate between images of distinct classes. During
testing, each target class is compared with either a genuine, i.e., target
class random sample, or an anomalous sample, i.e., from any class other than
the target class. The network outputs a relation score of the sample with the
target class sample. In datasets that contain genuine and forged samples of
each class, the method is altered. First, we train the network in the same
multi-class procedure mentioned above. Then, we further fine-tune the model by
making it learn to differentiate between genuine and forged samples of each
target class. The sample set contains genuine target class images, while the
query set has either a forged or genuine image from the combined forgery
dataset. The network outputs a relation score, and if the ground truth value
of the query is genuine, the error is computed between the score 0 and 1.
Similarly, if the ground truth is forged, the error is measured with 0. This
error is back-propagated as the loss in the unfrozen layers of the network.
For testing, the process is similar, where the support set contains genuine
samples, and the test set contains both forged and genuine samples that are to
be tested. The network is cross-validated after every 10 episodes, where the
weights yielding the highest accuracy are saved.

Figure 6: End to end architecture for biometric verification through our
approach. This approach encapsulates OC-SVM and a shallow neural network to
produce a fast, lightweight solution that can produce results that can be
compared to fully connected networks. The hand-picked features allows us to
visualize the feature that are being used for initial prediction.

##  5 Network Architecture

The proposed architecture combines a few-shot learning model with a feature
extraction module for a kernel-based, one-class classification technique. The
encoding module in the Relation Network consists of four convolutional blocks,
which is similar to most few-shot embedding modules, while the relation module
consists of two convolutional blocks. As designed by (Sung et al., 2018), each
block includes a 64-channel $3\times 3$ kernel convolution, batch
normalization, and ReLU activation. The first two blocks of each module also
contain a $2\times 2$ maxpooling layer. The blocks are followed by two fully-
connected linear layers with dimensions 8 and 1, respectively, with the last
uni-dimensional layer corresponding to the final output, that is, the relation
score for each query-sample pair. The output vector also passes through a
sigmoid function to ensure the scores lie in the range of (0,1). The data-
preprocessing timeline, as described above, is followed to create feature
vectors for the entire dataset before evaluation. During the testing phase,
while the Relation Network computes the relation score of the query image, the
feature vectors of the target class are fed into the OC-SVM to create a
decision boundary. The distance from this boundary to the test sample feature
vector, as determined by the OC-SVM, is scaled and mapped through an inverse
sigmoid function that outputs a confidence factor. This factor is realised as
a threshold value for the Relation Network output score. The network
architecture is summarized.

##  6 Experiments

The most prominent types of biometrics include signatures and periocular
region analysis. We will evaluate our technique in the aforementioned domains
through the following datasets: UBIPr (Padole and Proença, 2012), SigComp 2011
(Liwicki et al., 2011a), and SigWiComp 2013 (Malik et al., 2013)

In our experiments, the majority of absolute distances computed by the OC-SVM
appear to be concentrated between 0 and 0.3. Most values are very small and
close together, and can be used as inputs to our inverse sigmoid function. As
a result, the sigmoid function produces approximately the same threshold value
of approximately 0.5 most of the time. In this instance, user-specific
thresholding serves no purpose. To overcome this, we have scaled our OC-SVM
distances using a piecewise function. This function has two parts: for inputs
greater than —0.98—, it returns the same value, while inputs between
[-0.98,0.98] are fed into a scaled Tanh function. Furthermore, we have
included a multiplying factor of approximately ($\sim$2.36) in the Tanh
function to increase the range of output values. This multiplying factor also
ensures that both parts generate equal outputs when the inner limit values
($\pm$ 0.98) are input, resulting in a smoother transition between the two
parts. The output of this function is considered as an input to the
confidence-mapping inverse sigmoid function.

###  6.1 UBIPr

This dataset is a version of UBIRIS.v2 where the images are cropped in a way
that cover a wider part of the ocular region (Tax and Duin, 2004). This
dataset contains 10,252 RGB images captured by Canon EOS 5D Camera from 344
subjects with $(501\times 401)$ pixels of resolution. Images in this dataset
are taken from a distance of 4m to 8m to include distance variability. Due to
gaze and pose variation between $(0^{\circ},30^{\circ},-30^{\circ})$ and hair
covering the periocular region, this dataset suffers from the problem of
occlusion. One subject contained only a single image so we removed that from
our experiments.

####  6.1.1 Training

All input images were resized to ($128\times 128$) and normalized according to
the ImageNet statistics. The proposed architecture requires the combination of
two independent models: OC-SVM and Relation Network. The Relation Network is
trained episodically using 5 classes at a time (5-way), 1 sample image per
class (1-shot), and 25 query images; 5 for each class. This results in a total
of 30 embeddings per episode and 25 concatenated pairs being fed into the
relation network module, over 10,000 training episodes. For our data
preprocessing, we set the number of clusters to 100 during the development of
the bag-of-visual-words. The normalized histogram undergoes Principal
Component Analysis (PCA) to reduce redundant dimensions. This dataset of
extracted features is made ready before testing. The vector belonging to the
target class is fed to the OC-SVM to create a decision boundary that
encapsulates the known distribution.

####  6.1.2 Results

We combine our models to create a few-shot learning verification architecture.
Emulating Yilmaz et al. (2011), we determine the accuracy by averaging over
200 randomly generated episodes from the test set with 1 query image batched
per target class. We conduct testing by computing the relation or similarity
score of a random test sample with the genuine class sample. The randomly
selected test sample has an equal chance of being genuine or anomalous, where
all classes apart from the genuine target class are grouped together as
anomalous. We evaluate our model’s performance using two separate experiments:
in one, we compute relations between a single target class sample and the
query image, and in another, we assess the maximum relation score between the
query image and 5 samples per target class. This improves our accuracy via
more comparisons, reducing the impediment caused by occlusion. The results
achieved using this method are listed in Table 1, along with previous methods
for periocular image verification.

Table 1: UBIPr Results Comparison Model | Position | Accuracy | FAR | EER
---|---|---|---|---
Discriminitive CF (Smereka et al., 2016) |  | Left
---
Right
| 78.59%
---
84.14%
| 0.001%
---
0.001%
| 8.00%
---
6.46%
UBIPr CF(Smereka and Kumar, 2017) |  | Left
---
Right
| 77.95%
---
84.33%
| 0.001%
---
0.001%
| 8.93%
---
7.27%
PPDM (Smereka et al., 2015) |  | Left
---
Right
| 81.49%
---
85.28%
| 0.001%
---
0.001%
| 7.67%
---
7.09%
Proposed Model |  | Left
---
Right
| 91.01%
---
90.09%
| 0.001%
---
0.001%
| 4.49%
---
4.5%

###  6.2 SigComp 2011

The SigComp dataset contains both simultaneous online and off-line signature
images, but we only use off-line images for our purpose. The collection is
divided into two separate groups: Dutch and Chinese signatures. For the Dutch
set, the training set contains 362 offline signatures of 10 reference writers,
with 235 genuine signatures and 123 forgeries of these writers. The test set
contains a total of 1932 offline signatures of 54 reference writers, including
forgeries of them. The reference set contains 646 genuine signatures, while
1287 questioned signatures are provided for testing. The Chinese dataset has a
training set of 575 offline signatures of 10 reference writers and their
forgeries, containing 235 genuine signatures and 340 forgeries. The test set
for this subset contains 602 offline signatures of 10 reference writers and
their respective forgeries, with 115 referenced and 487 questioned signatures.
The images were collected using a WACOM Intuos3 A3 Wide USB Pen Tablet and
MovAlyzer software (Uzair et al., 2015). All input images are resized and
cropped to ($128\times 128$)and converted to 1 input channel.

####  6.2.1 Training

The data preprocessing and OC-SVM procedure are the same as mentioned in
Section 6.1.1. However, the Relation Network training method is modified.
First, we train the Relation Network using 5-way-5-shot with four query images
per class for classifying between genuine signatures of different subjects for
10,000 training episodes. Then, we further fine-tune the model over 1,000
episodes under one-class settings by learning to compare genuine and forged
samples of the same subject. In this approach, a query image is batched with a
target class sample, where the genuine query would have a ground truth value
of ‘1’, and a forgery would be classified as ‘-1’. This enables the model to
capture and comprehend the fine-grained differences in details between a
signature and its forgery.

####  6.2.2 Results

We evaluate the proposed architecture on the provided test set for each
subset. Testing is conducted by using the provided test reference set as a
support set and the provided questioned set, containing genuine and forged
signatures, as the test set. Each questioned image of a target class is
compared with either 1 or 5 genuine samples of the class under separate
experiments. The threshold-determining process works similarly, and the
results of the fused architecture are summarized in Tables 2 and 3 for the
Chinese and Dutch subsets, respectively. We compare the results against the
techniques used by other competitors during SigComp 2011.

Table 2: SigComp’11 Chinese Results Comparison (Liwicki et al., 2011b) Models | Accuracy | FAR | EER
---|---|---|---
Sabanci | 80.40% | 19.62% | 20.315%
Anonymous-1 | 73.10% | 26.70% | 2.10%
HDU | 72.90% | 26.98% | 27.24%
DFKI | 62.01% | 38.15% | 38.24%
VGG-16 (Alvarez et al., 2016) | 88% | 8.2% | 13.2%
Proposed Model | 75.60% | 7.60% | 12.20%
Table 3: SigComp’11 Dutch Results Comparison (Liwicki et al., 2011b) Models | Accuracy | FAR | EER
---|---|---|---
Sabanci | 82.91% | 16.41% | 17.17%
Qatar | 97.67% | 2.19% | 2.33%
HDU | 87.80% | 12.05% | 12.20%
DFKI | 75.84% | 24.57% | 24.17%
VGG-16 (Alvarez et al., 2016) | 94% | 13.32% | 8.22
Proposed Model | 76.83% | 3.30% | 11.55%

###  6.3 SigWiComp 2013

SigComp 2013 contains signatures from Japanese and Dutch writers in the same
format as SigComp 2011. We only use the off-line Japanese signatures to test
the multilingual signature verification capabilities of our architecture. The
publicly available Task SigJapanese contains only off-line signatures and has
a training set consisting of genuine and forged signatures from 11 distinct
subjects. Each subject has 42 genuine and 36 forged signatures, resulting in a
total of 858 signatures, of which 462 are genuine and 396 are forged. We
divide the subjects into eight training and three testing subjects, with our
model being trained on the genuine and forged signatures of eight subjects and
evaluated on the remaining three subjects. This leaves us with 624 images for
training and 234 images for testing.

####  6.3.1 Training

Our data pipeline preprocesses all signature images, and the resulting output
is then used for model evaluation. The Relation Network training procedure is
similar to that used for SigComp 2011, with 10,000 training episodes on
genuine data and an additional 5000 training episodes for fine-tuning using
forged and genuine samples.

####  6.3.2 Results

In the testing phase, we evaluate our fused architecture on the remaining
three subjects by averaging the accuracies over 200 episodes. Each query image
is compared with 1 or 5 genuine samples of the target class. The results are
summarized in Table 4.

Table 4: SigWiComp’13 Japanese Results Comparison Models | Accuracy | FAR | EER
---|---|---|---
Yilmaz (Yilmaz et al., 2011) | 90.72% | 9.72% | 9.73%
Kovari (Kovari and Charaf, 2013) | 72.70% | 27.36% | 27.29
Hassane (Hassaïne et al., 2012) | 66.67% | 33.33% | 33.33%
Djeddi (Djeddi et al., 2012) | 72.10% | 27.89% | 27.90%
Proposed Model | 92% | 0.001% | 4%

##  7 Conclusion

In conclusion, this study has demonstrated that our proposed approach, which
combines hand-crafted features and deep feature embeddings, can produce
results that are comparable to state-of-the-art models. Our fusion of the
Relation Network training method with the OC-SVM algorithm has enabled us to
implement a few-shot learning mechanism for biometric verification. By
thresholding the relational score for genuine classes with a sigmoid-mapped
distance from the hyperplane in the OC-SVM, we are able to strictly monitor
the false acceptance rate, which is critical from a biometric standpoint. The
obtained results suggest that our approach can be used to build effective and
efficient biometric verification systems. While there is still room for
improvement, we believe that our approach is a simple and convenient solution
that can be easily adopted in various applications. Future work could explore
the potential of incorporating other deep learning techniques or alternative
hand-crafted features to further improve the performance of our system.

## References

  * Alvarez et al. (2016) Gabe Alvarez, Blue Sheffer, and Morgan Bryant.  Offline signature verification with convolutional neural networks.  _Tech. Report_ , 2016.
  * Blanchard et al. (2010) Gilles Blanchard, Gyemin Lee, and Clayton Scott.  Semi-supervised novelty detection.  _Journal of Machine Learning Research_ , 11(99):2973–3009, 2010.  URL http://jmlr.org/papers/v11/blanchard10a.html.
  * Bradley (1997) Andrew P. Bradley.  The use of the area under the roc curve in the evaluation of machine learning algorithms.  _Pattern Recogn._ , 30(7):1145–1159, July 1997\.  ISSN 0031-3203.  doi: 10.1016/S0031-3203(96)00142-2.  URL https://doi.org/10.1016/S0031-3203(96)00142-2.
  * Caruana (1998) R. Caruana.  Multitask learning.  In _Encyclopedia of Machine Learning and Data Mining_ , 1998.
  * Djeddi et al. (2012) Chawki Djeddi, Labiba Souici-Meslati, and Abdellatif Ennaji.  Writer recognition on arabic handwritten documents.  In _Proceedings of the 5th International Conference on Image and Signal Processing_ , ICISP’12, page 493–501, Berlin, Heidelberg, 2012. Springer-Verlag.  ISBN 9783642312533.  doi: 10.1007/978-3-642-31254-0˙56.  URL https://doi.org/10.1007/978-3-642-31254-0_56.
  * Erfani et al. (2016) Sarah M. Erfani, Sutharshan Rajasegarar, Shanika Karunasekera, and Christopher Leckie.  High-dimensional and large-scale anomaly detection using a linear one-class svm with deep learning.  _Pattern Recognition_ , 58:121–134, 2016.  ISSN 0031-3203.  doi: https://doi.org/10.1016/j.patcog.2016.03.028.  URL https://www.sciencedirect.com/science/article/pii/S0031320316300267.
  * Erfani et al. (2017) Sarah M. Erfani, Mahsa Baktashmotlagh, Masud Moshtaghi, Vinh Nguyen, Christopher Leckie, James Bailey, and Kotagiri Ramamohanarao.  From shared subspaces to shared landmarks: A robust multi-source classification approach.  In _Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence_ , AAAI’17, page 1854–1860. AAAI Press, 2017.
  * Fei-Fei et al. (2006) Li Fei-Fei, R. Fergus, and P. Perona.  One-shot learning of object categories.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 28:594–611, 2006.
  * Finn et al. (2017) Chelsea Finn, Pieter Abbeel, and Sergey Levine.  Model-agnostic meta-learning for fast adaptation of deep networks.  In Doina Precup and Yee Whye Teh, editors, _Proceedings of the 34th International Conference on Machine Learning_ , volume 70 of _Proceedings of Machine Learning Research_ , pages 1126–1135. PMLR, 06–11 Aug 2017.  URL http://proceedings.mlr.press/v70/finn17a.html.
  * Görnitz et al. (2013) Nico Görnitz, Marius Kloft, Konrad Rieck, and Ulf Brefeld.  Toward supervised anomaly detection.  _J. Artif. Int. Res._ , 46(1):235–262, January 2013.  ISSN 1076-9757.
  * Hassaïne et al. (2012) Abdelâali Hassaïne, Somaya Al-Maadeed, and Ahmed Bouridane.  A set of geometrical features for writer identification.  In _Proceedings of the 19th International Conference on Neural Information Processing - Volume Part V_ , ICONIP’12, page 584–591, Berlin, Heidelberg, 2012. Springer-Verlag.  ISBN 9783642344992.  doi: 10.1007/978-3-642-34500-5˙69.  URL https://doi.org/10.1007/978-3-642-34500-5_69.
  * Hawkins et al. (2002) Simon Hawkins, Hongxing He, Graham J. Williams, and Rohan A. Baxter.  Outlier detection using replicator neural networks.  In _Proceedings of the 4th International Conference on Data Warehousing and Knowledge Discovery_ , DaWaK 2000, page 170–180, Berlin, Heidelberg, 2002. Springer-Verlag.  ISBN 3540441239.
  * Hodge and Austin (2018) Victoria J Hodge and Jim Austin.  Deep hybrid learning: A new frontier in anomaly detection.  _IEEE Computational Intelligence Magazine_ , 13(3):47–56, 2018.  doi: 10.1109/MCI.2018.2842197.
  * Jia et al. (2014) Y. Jia, Evan Shelhamer, J. Donahue, S. Karayev, J. Long, Ross B. Girshick, S. Guadarrama, and Trevor Darrell.  Caffe: Convolutional architecture for fast feature embedding.  _Proceedings of the 22nd ACM international conference on Multimedia_ , 2014.
  * Kovari and Charaf (2013) Bence Kovari and Hassan Charaf.  A study on the consistency and significance of local features in off-line signature verification.  _Pattern Recogn. Lett._ , 34(3):247–255, February 2013.  ISSN 0167-8655.  doi: 10.1016/j.patrec.2012.10.011.  URL https://doi.org/10.1016/j.patrec.2012.10.011.
  * Krishnapriya et al. (2020) K. S. Krishnapriya, Vítor Albiero, Kushal Vangara, Michael C. King, and Kevin W. Bowyer.  Issues related to face recognition accuracy varying based on race and skin tone.  _IEEE Transactions on Technology and Society_ , 1(1):8–20, 2020.  doi: 10.1109/TTS.2020.2974996.
  * Krizhevsky et al. (2012) Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton.  Imagenet classification with deep convolutional neural networks.  In _Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1_ , NIPS’12, page 1097–1105, Red Hook, NY, USA, 2012. Curran Associates Inc.
  * Lake et al. (2011) B. Lake, R. Salakhutdinov, J. Gross, and J. Tenenbaum.  One shot learning of simple visual concepts.  _Cognitive Science_ , 33, 2011.
  * Liwicki et al. (2011a) M. Liwicki, Muhammad Imran Malik, C. E. V. D. Heuvel, X. Chen, C. Berger, R. Stoel, M. Blumenstein, and B. Found.  Signature verification competition for online and offline skilled forgeries (sigcomp2011).  _2011 International Conference on Document Analysis and Recognition_ , pages 1480–1484, 2011a.
  * Liwicki et al. (2011b) Marcus Liwicki, Muhammad Imran Malik, C. Elisa van den Heuvel, Xiaohong Chen, Charles Berger, Reinoud Stoel, Michael Blumenstein, and Bryan Found.  Signature verification competition for online and offline skilled forgeries (sigcomp2011).  In _2011 International Conference on Document Analysis and Recognition_ , pages 1480–1484, 2011b.  doi: 10.1109/ICDAR.2011.294.
  * Malik et al. (2013) Muhammad Imran Malik, Marcus Liwicki, Linda Alewijnse, Wataru Ohyama, Michael Blumenstein, and Bryan Found.  Icdar 2013 competitions on signature verification and writer identification for on- and offline skilled forgeries (sigwicomp 2013).  In _2013 12th International Conference on Document Analysis and Recognition_ , pages 1477–1483, 2013.  doi: 10.1109/ICDAR.2013.220.
  * Miller et al. (2016) Alexander H. Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Antoine Bordes, and J. Weston.  Key-value memory networks for directly reading documents.  In _EMNLP_ , 2016.
  * Ojala et al. (2002) T. Ojala, M. Pietikainen, and T. Maenpaa.  Multiresolution gray-scale and rotation invariant texture classification with local binary patterns.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 24(7):971–987, 2002.  doi: 10.1109/TPAMI.2002.1017623.
  * Ojansivu and Heikkilä (2008) Ville Ojansivu and Janne Heikkilä.  Blur insensitive texture classification using local phase quantization.  In Abderrahim Elmoataz, Olivier Lezoray, Fathallah Nouboud, and Driss Mammass, editors, _Image and Signal Processing_ , pages 236–243, Berlin, Heidelberg, 2008. Springer Berlin Heidelberg.  ISBN 978-3-540-69905-7.
  * Padole and Proença (2012) C. Padole and Hugo Proença.  Periocular recognition: Analysis of performance degradation factors.  _2012 5th IAPR International Conference on Biometrics (ICB)_ , pages 439–445, 2012.
  * Patil and Banyal (2019) Ganesh G. Patil and R. K. Banyal.  Techniques of deep learning for image recognition.  In _2019 IEEE 5th International Conference for Convergence in Technology (I2CT)_ , pages 1–5, 2019.  doi: 10.1109/I2CT45611.2019.9033628.
  * Perera and Patel (2019) Pramuditha Perera and Vishal M. Patel.  Learning deep features for one-class classification.  _IEEE Transactions on Image Processing_ , 28(11):5450–5463, 2019.  doi: 10.1109/TIP.2019.2917862.
  * Rezende et al. (2016) Danilo Jimenez Rezende, S. Mohamed, Ivo Danihelka, K. Gregor, and Daan Wierstra.  One-shot generalization in deep generative models.  In _ICML_ , 2016.
  * Rublee et al. (2011) Ethan Rublee, V. Rabaud, K. Konolige, and G. Bradski.  Orb: An efficient alternative to sift or surf.  _2011 International Conference on Computer Vision_ , pages 2564–2571, 2011.
  * Schölkopf et al. (1999) Bernhard Schölkopf, Robert Williamson, Alex Smola, John Shawe-Taylor, and John Platt.  Support vector method for novelty detection.  In _Proceedings of the 12th International Conference on Neural Information Processing Systems_ , NIPS’99, page 582–588, Cambridge, MA, USA, 1999\. MIT Press.
  * Shubathra et al. (2020) S Shubathra, PCD Kalaivaani, and S Santhoshkumar.  Clothing image recognition based on multiple features using deep neural networks.  In _2020 International Conference on Electronics and Sustainable Communication Systems (ICESC)_ , pages 166–172, 2020.  doi: 10.1109/ICESC48915.2020.9155959.
  * Smereka and Kumar (2017) Jonathon Smereka and B. Kumar.  _Identifying the best periocular region for biometric recognition_ , pages 125–147.  Security. Institution of Engineering and Technology, 07 2017.  doi: 10.1049/PBSE005E˙ch6.  URL https://digital-library.theiet.org/content/books/10.1049/pbse005e_ch6.
  * Smereka et al. (2015) Jonathon M. Smereka, Vishnu Naresh Boddeti, and B. V. K. Vijaya Kumar.  Probabilistic deformation models for challenging periocular image verification.  _IEEE Transactions on Information Forensics and Security_ , 10(9):1875–1890, 2015.  doi: 10.1109/TIFS.2015.2434271.
  * Smereka et al. (2016) Jonathon M. Smereka, B. V. K. Vijaya Kumar, and Andres Rodriguez.  Selecting discriminative regions for periocular verification.  In _2016 IEEE International Conference on Identity, Security and Behavior Analysis (ISBA)_ , pages 1–8, 2016.  doi: 10.1109/ISBA.2016.7477247.
  * Sukhbaatar et al. (2015) Sainbayar Sukhbaatar, Arthur D. Szlam, J. Weston, and R. Fergus.  End-to-end memory networks.  In _NIPS_ , 2015.
  * Sung et al. (2018) Flood Sung, Yongxin Yang, L. Zhang, Tao Xiang, P. Torr, and Timothy M. Hospedales.  Learning to compare: Relation network for few-shot learning.  _2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 1199–1208, 2018.
  * Tax and Duin (2004) David Tax and Robert Duin.  Support vector data description.  _Machine Learning_ , 54:45–66, 01 2004.  doi: 10.1023/B:MACH.0000008084.60811.49.
  * Uzair et al. (2015) M. Uzair, A. Mahmood, A. Mian, and Chris McDonald.  Periocular region-based person identification in the visible, infrared and hyperspectral imagery.  _Neurocomputing_ , 149:854–867, 2015.
  * Vinyals et al. (2016) Oriol Vinyals, C. Blundell, T. Lillicrap, K. Kavukcuoglu, and Daan Wierstra.  Matching networks for one shot learning.  In _NIPS_ , 2016.
  * Wang et al. (2020) Yaqing Wang, Quanming Yao, James T. Kwok, and Lionel M. Ni.  Generalizing from a few examples: A survey on few-shot learning.  _ACM Comput. Surv._ , 53(3), June 2020.  ISSN 0360-0300.  doi: 10.1145/3386252.  URL https://doi.org/10.1145/3386252.
  * Wu et al. (2015) Ruobing Wu, Baoyuan Wang, Wenping Wang, and Yizhou Yu.  Harvesting discriminative meta objects with deep cnn features for scene classification.  In _2015 IEEE International Conference on Computer Vision (ICCV)_ , pages 1287–1295, 2015.  doi: 10.1109/ICCV.2015.152.
  * Yilmaz et al. (2011) Mustafa Berkay Yilmaz, Berrin Yanikoglu, Caglar Tirkaz, and Alisher Kholmatov.  Offline signature verification using classifier combination of hog and lbp features.  In _2011 International Joint Conference on Biometrics (IJCB)_ , pages 1–7, 2011.  doi: 10.1109/IJCB.2011.6117473.
  * Zhu et al. (2018) Z. Zhu, Guicang Zhang, and H. Li.  Surf feature extraction algorithm based on visual saliency improvement.  _International Journal of Engineering and Applied Sciences_ , 5, 2018\.