# Palmprint image registration using convolutional neural networks and Hough
transform

Mohsen Ahmadi, Hossein Soleimani

###### Abstract

Minutia-based palmprint recognition systems got lots of interest in the last
two decades. Due to a large number of minutiae in a palmprint, approximately
1000 minutiae, the matching process is time-consuming which makes it
unpractical for real-time applications. One way to address this issue is
aligning all palmprint images to a reference image and bringing them to a same
coordinate system. Bringing all palmprint images to a same coordinate system,
results in fewer computations during minutia matching. Using convolutional
neural network (CNN) and generalized Hough transform (GHT), we propose a new
method to register, align, palmprint images accurately. This method, finds the
corresponding rotation and displacement (in both $x$ and $y$ direction)
between the palmprint and a reference image. Proposed method is capable of
distinguishing between left and right palmprint automatically which helps to
speed up the matching process. Furthermore, designed structure of CNN in the
registration stage gives us the segmented palmprint image which is a pre-
processing step for minutia extraction. The proposed registration method
followed by minutia-cylinder code (MCC) matching algorithm has been evaluated
on the THUPALMLAB database, and the results show the superiority of our
algorithm over most of the state-of-the-art algorithms.

###### Index Terms:

Palmprint Matching, Image Registration, CNN, Hough Transform.

##  1 Introduction

Biometrics is used to recognize or verify human identity based on physical or
behavioral characteristics. Biometric features such as face, iris,
fingerprint, hand geometry, palmprint and signature have been used for human
identification and recognition [1]. Among all these features, recently,
palmprint recognition has gained considerable attention as a reliable personal
identification technique. Palmprint inherits many of the fingerprint features.
Both of them are represented by the information presented in a friction ridge
impression including ridge flow, ridge characteristics, and ridge structure.
Due to their uniqueness and permanence, palmprint and fingerprint
identification has been generally trusted [2], [3].

In palmprint recognition systems mainly two different types of images: 1) low
resolution, i.e. less than $200$ dpi, and 2) high resolution ($500$ dpi)
images are used [3], [4]. Features in the low resolution palmprints includes
principal lines, wrinkles and ridges. These features define the palmprint as a
texture containing discriminatory features which are relatively stable and
applicable in biometric identification or verification systems [5], [6], [7].
In low-resolution palmprint images, ridges are not seen, and it is not
feasible to extract the second-level information, which is more discriminant
[8]. Features obtained from low resolution images are more suitable for civil
and commercial application such as access control [3]. On the other hand,
features obtained from high resolution images are minutiae, the orientation
image, singular points, and the ridge frequency image. High resolution images
are suitable for forensic application such as criminal detection [9]. Because
these features are robust against time passing, they are considered admissible
in a court of law [10]. In this paper, our focus will be on high resolution
palmprint images and minutia-based palmprint matching. Our designed
registration method is based on first-level information (mainly orientation
image of palmprint), and it is applicable as long as the palmprint images are
high resolution .

In literature, a lot of effort has been made regarding minutia-based palmprint
matching in order to reduce the matching time and to address other challenges.
In [11], similarity between two palmprints is calculated by means of a
weighted sum of minutiae and orientation field matching scores. Li et.al
describes a palmprint matching algorithm where minutiae are compared by means
of a local and global matching [12]. Multi-feature-based palmprint recognition
system is another strategy which is suggested in [8]. Minutia, orientation
image, density map, and major creases are extracted to achieve higher matching
accuracy. In [13], a new coarse-to-fine palmprint algorithm is proposed to
deal with the large number of minutiae in palmprints. A clustering algorithm
is applied to separate minutiae into different groups and make the matching
process much faster. Raffaele and his colleagues, in [14], proposed a matching
algorithm which is robust against skin distortion. They apply two stage
minutia-based matching: 1) local minutia matching using Minutia Cylinder-Code
(MCC) algorithm [15] and 2) new global score computation.

Use of spectral minutiae representation and segmenting palmprint into three
different regions is another approach which aims to design faster matching
algorithm[16]. In [17] ridge distance and conventional minutia descriptors are
used together to speed up the matching. The difference of ridge distance near
the minutiae is calculated, and only the pairs with similar ridge distance are
considered. Hence, the number of minutiae comparison is greatly reduced which
results in less time complexity. In [18], a novel region-quality based minutia
extraction algorithm is proposed to improve the accuracy of palmprint
recognition. Also, authors proposed an efficient and minutiae based encoding
and matching algorithm to accelerate the palmprint matching.

Despite the available algorithms, some problems with the palmprint matching
still need to be solved for large-scale applications [19]: skin distortion,
diversity of different palm regions, and computational complexity. Registering
palmprint images into the same coordinate, has been used to address the skin
distortion and problem of large number of minutia comparison during the match
process [19], [20]. By registration, it does not need to compare all minutia
from two pamprint. Therefore we need to apply the matching algorithm only to
the same parts of the two palmprints. This helps to mitigate the effects of
skin distortion and improve matching speed. However, the registration methods
proposed in [19], [20] are not accurate enough, and also for algorithm in [19]
there is no criterion to show whether the registration is successful.

To increase the matching speed and accuracy, motivated by [19] and our
previous work [20], we developed a new palmprint registration method. In this
method we combine CNN and GHT to register all palmprint images with a
reference image. Using CNN, we first roughly find the rotation difference
between the current pamprint and the reference palmprint. Then, we find the
exact rotation difference and translation between these two images by GHT. Two
new criteria are used to measure the confidence of the registration. Also,
using these two criteria our method is capable of recognizing left palm from
right palm which can double the matching speed if we do not know whether the
current palm is coming from the left hand or right hand. Furthermore, the
designed structure of CNN in registration stage, gives us the segmented
palmprint image from background which is a pre-processing step for minutia
extraction.

Our minutiae matching stage is based on the MCC method with some modification
[15], which includes a local matching stage based on MCC, followed by a
relaxation procedure to compute a global matching score.

By proposed method, the matching time is much better than the state-of-the-
art. On the other hand, because of accurate registration, we could subside the
distortion effect, and as a result the matching accuracy improved. The
experimental results on THUPALMLAB database [21] indicate that the proposed
system achieves a false non-match rate (FNMR) of $0.24\%$ while the false
match rate (FMR) is controlled at $0$. As for the speed of our system, the
average time of full palmprint matching is about $6$ milliseconds. The rest of
this paper is organized as follows: Section 2 outlines the details of the
proposed palmprint registration process. In section 3 matching strategy is
explained. In Section 4, the experimental results are presented and analyzed.
Finally, we end up with conclusions in section 5.

##  2 Proposed palmprint Registration

Most of the raw palmprint images are not in the same coordinate system. By
applying registration and bringing images to same coordinate system, the
matching process will be facilitated. By registering, we need to compare only
minutiae in both the query and gallery image which have almost the same
location and direction, and this accelerates the matching process.

Most of the palmprint registration methods are based on intervals between
fingers [22], [23] or hand contour and principle lines [24]. These features
are suitable for low resolution images which are captured by contact-less
devices, ensuring the whole palm region and the finger roots are visible.
However, in high resolution images fingers are not seen or hand contour is
incomplete and unreliable. Orientation fields of different palms are quite
similar, but the orientation of different palm regions is distinctive. This
makes the orientation field to be a reliable feature for palmprint
registration.

In [19], [20], the average orientation field of palmprints has been used as
the reference image, and other palmprints are aligned with this image.
Parameters (rotation around $z$ axis and displacement in $x$ and $y$
direction) of rigid registration between reference orientation field and the
estimated orientation field of each palmprint are obtained. This
transformation (rigid body) is applied to unregistered images and brings them
to the same coordinate system. These methods fail when the rotation difference
between the reference image and the floating image is large (greater than $90$
degrees). The situation is much worse when the palmprint image is not that
high quality, and as a result the estimated orientation field is not reliable.
To address this issue, we first roughly find the rotation difference between
two images by CNN. CNN does not need to extract orientation field, and it is
robust against distortion and low quality. After that, the exact values of
registration parameters is obtained by use of GHT. Figure 1 shows the steps of
proposed registration method.

Figure 1: Steps of proposed palmprint registration method

###  2.1 Registration by CNN

To find the rotation of a plamprint image around $z$ axis, CNN is applied. Raw
palmprint image is fed to CNN, and the output will show us the rotation of the
image around $z$ axis. We formulated this task as a classification problem,
i.e., the input palmprint will be classified to one of the $24$ existing
classes. Theses classes are $0^{\circ},15^{\circ},...,345^{\circ}$. The
architecture of designed CNN is shown in figure 2.

Figure 2: Architecture of designed CNN

It consists of $8$ convolutional layers, each layer includes convolution and
max-pooling steps, followed by a fully connected layer and a soft-max function
to calculate the final probabilities of each class. Kernel size in first layer
is $9\times 9$ and it changes to $3\times 3$ in other convolutional layers. In
the proposed network, as illustrated in figure 2, $16$, $24$, $32$, $48$,
$64$, $96$, $128$, and $256$ kernels are used for Conv layer $1$, $2$, $3$,
$4$, $5$, $6$, $7$ and $8$ respectively, with stride $1$. For pooling layers,
kernel size is set to $3\times 3$, and stride $2$. We also applied dropout ,
Batch-Normalization [25] and extensive data augmentation to overcome over
fitting. Rectified Linear Unit (Relu) activation function is widely used in
DNN. Because of Dying Relu problem, we applied exponential linear unit (ELU)
[26] activation function instead of Relu function.

We used $500$ palm images in training phase, $300$ palms randomly selected
from THUPALMLAB database [21] and $200$ palms acquired by a live scanner. All
these $500$ palms were rotated around $z$ axis such that their rotation is
almost $0^{\circ}$ rspect to $y$ axis. To build the augmented training data,
images were rotated with random values from
$\\{0^{\circ},15^{\circ},...,345^{\circ}\\}$ set respect to $y$ axis. For each
image, we produced $20$ different images with different random rotations, and
each new image with its rotation information was saved in data set. Therefore,
the training data contains $500*20=10000$ palmprint images. To have a much
generalized data set, other augmentation techniques such shrinkage, adding
noise and flipping respect to $y$ axis were applied.

CNN output is vector $Y$ with length $24$ showing the probabilities of each
$24$ classes. The class with highest probability shows the rotation of input
palmprint image respect to $y$ axis. For instance, if the probability of class
$i,i=0,1,...,23$ is maximum, then the rotation around $z$ axis and respect to
$y$ axis is $(i\times 15)^{\circ}$. To use CNN and find the rotation, for a
given input image we smooth the output vector $Y$ by a window with length $3$
and call it $Y_{s}$. Then, maximum value of vector $Y_{s}$ is considered,
$M_{s}$ . If $M_{s}>0.35$, then the output of network is reliable. Those
images that satisfies this condition, are rotated around $z$ axis by
$-(i*15)^{\circ}$ (correcting rotation, see figure 1) and are passed to the
second round of registration. On the other hand, images with smaller value of
$M_{s}$ are passed directly to feature extraction step; more likely
registration will fail if we pass these images to “registration refining”. For
these palm images we do not apply registration, and treat them differently in
local matching step (section 3.1).

Another advantage of designed CNN is its ability to segment palm region from
the background. We noticed that in trained model, the outputs of layer #3 are
near binary image which carry high values for regions with palm, and they near
to zero for pixels belonging to the the background. In other word, during the
training phase, the coefficient of filters of this layer were tuned in a way
that they give the segmented palmprint image. Figure 3 illustrates the
resulted mask image produced by combination of outputs of layer #3 for a
sample palmprint. As it is seen, it can be used to segment the foreground from
background. Palmprint segmentation is a pre-step before minutia extraction.
Now, we have this image without any extra computation.

Figure 3: Mask image produced from outputs of layer #3 of designed CNN

###  2.2 Registration refining by GHT

In previous section, by assigning each palmprint to one of $24$ classes of
angles, we roughly found the rotation difference between palmprint images
respect to ground truth. To find exact values of registration (displacement in
$x$ and $y$ directions and rotation around $z$ axis), we apply GHT. We know
that rotation around $z$ axis can not be larger than
${\frac{360}{24}}=15^{\circ}$ (we already corrected the rotation difference,
$\theta$ ). This will help us to find the exact value of $\theta$ much faster.
CNN outputs ($M_{s}$) are used to judge the level of confidence of
registration. If it is larger than a threshold, it means that the estimated
$\theta$ is reliable. For images with small values of $M_{s}$ we can not trust
the estimated $\theta$, and as a result we ignore doing registration on these
images (see figure 1).

Similar to [20], we used orientation field of each palmprint image to find the
registration parameters. The average orientation field of palmprints is used
as the reference image, and other palmprint are aligned with this image. The
proposed orientation field estimation algorithm in [20] is used to compute the
orientation field in each $16\times 16$ blocks. Finally, the GHT algorithm
[27], is applied to find registration parameters.

All possible pairs of $16\times 16$ blocks between the input (unregistered)
palmprint orientation and the reference orientation field vote for
corresponding rotation and the displacement (in $x$ and $y$ directions).
Contrary to [19] which uses inverse of circular standard deviation as weights
of each block in voting system, we calculate the weight of each block based on
its quality [20].

Palmprint registration, brings palm images to the same coordinate, which helps
to speed up the matching, with a small accuracy reduction. We need a criterion
to judge the result of the registration and to check if it can be trusted or
not. To measure the reliability of registration results, we define few new
parameters:

  * •

$q_{1}=\frac{\\#of\>all\>blocks\>in\>foreground}{\\#of\>all\>blocks\>in\>image}$

  * •

$q_{2}=\frac{\\#\>blocks\>vote\>for\>best\>candidate\>(\theta,d_{x},d_{y})}{\\#of\>all\>blocks\>in\>foreground}$

$q_{1}$ shows the percentage of blocks in an image which belong to palmprint,
i.e., foreground. A larger value of $q_{1}$ shows the higher quality of a
palmprint or it could also indicate larger palmprint image. Small values of
$q_{1}$ show that the palmprint image is not a full palmprint (latent
palmprint) or the palmprint image has low quality. If this value is less than
a threshold (in our case $0.3$), we ignore registration for that palmprint.
$q_{2}$ shows the quality of registration, and is the ratio of blocks which
vote for final registration parameters to all blocks in foreground. $q_{2}$
with larger values guarantees the registration accuracy; $q_{2}>0.5$ is
acceptable. These parameters can be used to determine if the input palmprint
is for right hand or left hand.

The reference orientation palmprint which we use for refining the registration
, belongs to left hand. GHT is applied to do exact registration for both
original and flipped version of the input palmprint (flipped around $y$ axis).
Now, two $q_{2}$s are in hand, resulted from registration of reference image
with orientation of input palmprint and its flipped version. If $q_{2}$ of
original palmprint is larger than $q_{2}$ of flipped image, then input
palmprint is left hand and vice versa. Note that, if the larger $q_{2}$ is
less than $0.5$, then the registration has most likely failed. Therefore, our
algorithm can automatically recognize left palmprint from right palmprint
which will increase the palmprint matching speed and accuracy.

Figure 4: Results of final registration for both original(left) and flipped
image(right). For both images $q_{1}=0.36$, for original image $q_{2}=0.3$ and
for flipped version $q_{2}=0.89$.  Figure 5:  Sample palmprint registered by
proposed method. First row shows the original images and the second raw
indicates the registered images

(a) Original image

(b) Registered by CNN

(c) Registered by GHT

(d) Enhanced image

(e) Skeleton image

(f) Minutiae

Figure 6: Sample output of each step of whole procedure

Figure 4 shows results of registration for a sample right hand palmprint.
$q_{1}=0.36$ satisfies the threshold while $q_{2}$ for left hand palmprint is
less than $0.5$ and it is much smaller than $q_{2}=0.89$ of right hand
palmprint. Therefore, features (Minutia) are extracted from right image, and
palmprint class is known, i.e., the palmprint belongs to right hand.

We applied the proposed registration strategy to THUPALMLAB database(train
set) and only a few of the images did not satisfy $q_{2}>0.5$. Maximum allowed
rotation was limited to $30^{\circ}$ (with steps=$3^{\circ}$) and maximum
displacement was restricted to $500$ pixels, which is $\frac{1}{4}$ of the
image width and height. For palmprints that $q_{1}$ or $q_{2}$ do not satisfy
thresholds, palmprint class is “unknown”.

Figure 5 shows some raw palmprint samples, from the same hand, and their
registered version. As it is seen in the second row, all three images are
almost in the same coordinate.

##  3 Palmprint Matching

After registration, the segmented palmprint is enhanced by applying Gabor
filters [28]. The enhancement, which is guided by the estimated local
orientations and frequencies, produces a near binary image which is then
simply binarized. The thinning algorithm in [29] is applied to the enhanced
binary image and the palmprint skeleton is obtained. Using the algorithm in
[2] minutiae are extracted and the results are refined by approach in [30].
Once the minutiae have been extracted, the matching process can be done.
Figure 6 shows the whole procedure for sample palmprint image.

The matching algorithm is similar to what we have done in [20]. It consists of
two stages: a) local minutia matching and b) global score calculation. In the
local minutia matching stage, the similarity of a minutia pairs is computed
with the MCC descriptor to make the matching process more efficient and fast.
In the second stage, overall similarity of two palmprints is calculated.

###  3.1 Local matching

The MCC descriptor is a local data structure which is invariant for
translation and rotation. This structure encodes spatial and directional
relationships among the minutia and its (fixed-radius) neighborhood. It is
represented by a bit vector of fixed length. To check the similarity of two
local minutiae from two different images, we compare their MCC descriptors.
For $N_{S}=16$ (number of cells along the cylinder diameter) and $N_{D}=5$
(number of cylinder section), each MCC descriptor contains a vector with
length equal to $16\times 16\times 5=1280$ bits. To check the similarity of
two minutiae, we need to do a $XOR$ operation between two vectors, with length
$1280$, which is a time consuming process and may not be suitable for
palmprint images. To decrease the number of similarity computation, we use
registered images. In other word, we only check the MCC similarity between two
minutia which roughly belong to the same part of the palm. By decreasing the
number of MCC similarity computations, the matching process will speed up. For
a given minutia $m_{A}(x_{mA},y_{mA},\theta_{mA})$ from palmprint A and
minutia $m_{B}(x_{mB},y_{mB},\theta_{mB})$ from palmprint B; If $m_{A}$ and
$m_{B}$ do not belong to the same region, their local similarity is zero. This
means, if $|x_{mA}-x_{mB}|>T_{x}$, $|y_{mA}-y_{mB}|>T_{y}$ or
$\theta_{mA}-\theta_{mB}>T_{rot}$ their MCC similarity is set to zero. For
images which fail to be registered by CNN,
$T_{rot}=180^{\circ},T_{x}=T_{y}=2048$. Images with small values of $q_{1}$ or
$q_{2}$ in second phase of registration(refining by GHT), $T_{rot}$,$T_{x}$
and $T_{y}$ are set to $30^{\circ}$, $1600$ and $1600$ respectively. Finally,
for images which are successfully registered,
$T_{rot}=20^{\circ},T_{x}=T_{y}=500$ (see figure 1).

Without registration , the original MCC algorithm needs to compare $N_{A}$
minutiae from image $A$ with $N_{B}$ minutiae from image $B$, where $N_{A}$
and $N_{B}$ are number of minutia in image $A$ and $B$, respectively. This
amount of comparison is time consuming and use of registration decrease this
complexity, dramatically.

###  3.2 Global matching

In order to compare two palmprints, a global score (denoting their overall
similarity) needs to be derived from the local similarities. The approach
proposed in [20] is used to calculate the global score. For two given MCC
minutia descriptor sets ${\bf
A}=\\{{MCC_{a_{1}},MCC_{a_{2}},...,MCC_{a_{N_{A}}}}\\}$ and ${\bf
B}=\\{{MCC_{b_{1}},MCC_{b_{2}},...,MCC_{b_{N_{B}}}}\\}$, there is a local
similarity score matrix, $S_{N_{A}\times N_{B}}$, showing the MCC similarity
between minutiae of image $A$ with those of image $B$. $S(i,j)$ is the MCC
similarity of minutia $i$ from image $A$ with minutia $j$ from image $B$.
Based on the section 3.1, $S$ is a low density matrix; most of its elements is
zero.

To compute the global score, $n_{R}$ minutia pairs with maximum similarity
values are selected. Suppose
$P=\\{(r_{t},c_{t}),0<r_{t}<N_{A}\>\>and\>\>0<c_{t}<N_{B}\\}$ is the selected
$n_{R}$ minutiae-index pairs. First $\hat{S}$, normalized version of $S$, is
computed:

$\hat{S}(i,j)=\langle 1-{{\sum^{N_{A}}_{\begin{array}[]{c}k=0\\\ k\neq
i\end{array}}S(k,j)+\sum^{N_{B}}_{\begin{array}[]{c}k=0\\\ k\neq
j\end{array}}S(i,k)}\over{N_{A}+N_{B}-2}}\rangle S(i,j).$ (1)

The normalization modifies each value according to the average of the values
in the same row and in the same column. After normalization, $n_{R}$ minutiae
pairs, corresponding to the top $n_{R}$ values in $\hat{S}$ matrix, are
selected and a relaxation procedure is applied to the $n_{R}$ similarity
scores guided by their corresponding minutia pairs.

Let $\hat{s}^{0}_{t}=\hat{S}(r_{t},c_{t})$ is initial normalized similarity of
pair $t$; the similarity of pair $t$ at iteration $i$ of the relaxation
procedure is:

$\hat{s}^{i}_{t}=w_{R}.\hat{s}^{i-1}_{t}+(1-w_{R}).{{\sum^{n_{R}}_{k=1,k\neq
t}\rho(t,k).\hat{s}^{i-1}_{k}}\over{n_{R}-1}},$ (2)

where $w_{R}\in[0,1]$ is a weighting factor and:

$\qquad\rho(t,k)=\prod_{i=1}^{3}Z(d_{i},\mu^{\rho}_{i},\tau^{\rho}_{i}),$

$\begin{array}[]{c}\\!d_{1}={{|d_{s}(a_{r_{t}},a_{r_{k}})-d_{s}(b_{c_{t}},b_{c_{k}})|}\over{max(d_{s}(a_{r_{t}},a_{r_{k}}),d_{s}(b_{c_{t}},b_{c_{k}}))}},\\\
\quad
d_{2}=|d_{\varphi}(d_{\theta}(a_{r_{t}},a_{r_{k}}),d_{\theta}(b_{c_{t}},b_{c_{k}}))|,\\\
\quad
d_{3}=|d_{\varphi}(d_{R}(a_{r_{t}},a_{r_{k}}),d_{R}(b_{c_{t}},b_{c_{k}}))|,\end{array}$
(3)

where $d_{s}(m_{1},m_{2})$ shows the Euclidean distance between minutia
$m_{1}$, $m_{2}$ and $d_{\varphi}(\theta_{1},\theta_{2})$ is the difference
(modulo $2\pi$) between two angles $\theta_{1}$, $\theta 2$.
$d_{\theta}(m_{1},m_{2})$ indicates directional difference between two
minutiae ($d_{\theta}(m_{1},m_{2})=d_{\varphi}(\theta_{1},\theta_{2})$).
$\rho(t,k)$ measures compatibility between two minutiae pairs
$(a_{r_{t}},a_{r_{k}})$ from image A and $(b_{c_{t}},b_{c_{k}})$ from image B.
It is computed as the product of three feature, $d_{1},d_{2}$ and $d_{3}$ and
normalized by a sigmoid functions with its specific $\mu$ and $\tau$:

$Z(\nu,\mu,\tau)={{1}\over{1+e^{-\tau(\nu-\mu)}}},$ (4)

$d_{1}$ computes distance similarity of two minutia pairs. $d_{2}$ is the
similarity of directional differences and $d_{3}$ denotes the radial angle
similarity. Radial angle is the angle between line connecting two minutiae and
the direction of the first minutia. The smaller $d_{1}$, $d_{2}$ and $d_{3}$
results in a smaller similarity value between two minutia pairs
$\\{(a_{r_{t}},a_{r_{k}}),(b_{c_{t}},b_{c_{k}})\\}$.

Relaxation procedure (2) is repeated for $n_{rel}$ times and an efficiency of
each pair $t$ is computed as follow:

$\varepsilon_{t}={{\hat{s}^{n_{rel}}_{t}}\over{\hat{s}^{0}_{t}}}.$ (5)

At the end, global score of two palmprint images A and B is calculated as the
average of the $n_{P}$ similarity values, $\hat{s}^{n_{rel}}_{t}$,
corresponding to top $n_{P}$ pairs with highest efficiency values.

##  4 Experimental results

In this section, experiments were carried out to evaluate the proposed method.
The proposed method is compared with other palmprint matching algorithms, and
results and parameters are reported.

###  4.1 Database and Parameter Tuning

The only publicly available high-resolution palmprint database, Tsinghua
Palmprint Database [21], is used as the main database. It consists of $1280$
palmprint images from $80$ different people (left and right palms and eight
impressions per palm). All images are $2040\times 2040$ pixels with $500$ dpi
resolution and 256 gray levels. By zero padding, the image size is changed to
$2048\times 2048$ ($2048$ is divisible by $16$ which is block-size in
orientation field estimation algorithm). The THUPALMLAB database consists of
two separate sets: 1) the training set and 2) the test set. The training set
(the first $20$ subjects) includes $320$ palmprints (from $40$ different
palms), while the test set contains the remaining $960$ palmprints (from $120$
different palms).The training set is used to tune the parameters, and it is
done by minimizing Equal Error Rate (EER).

Thanks to the registration, the MCC similarity of small percent of all
possible minutia pairs is non-zero, i.e, matrix $S$ is sparse. In the global
score computation phase, $n_{R}$ minutia pairs with maximum local similarity
score are selected from $\hat{S}$ Matrix. Therefore, the remaining non-zero
elements of the similarity matrix $S$ are more reliable, and the relaxation
phase can be done even with small values of $n_{R}$. Reducing $n_{R}^{max}$
from $300$(in original MCC) to $70$ decreases the relaxation phase
computations dramatically. With $n_{R}^{max}=300$, $\rho(t,k)$ needs to be
calculated almost $300\times 300=90000$ times in each iteration and this value
is much bigger than $70\times 70=4900$. In contrast to [14] where $N_{S}=16$,
in our proposed system, $N_{S}$ is $8$ and the resulted MCC descriptor length
is $8\times 8\times 5=320$ bits, and it takes much less time to compute the
MCC similarity of two minutiae. Also, since palmprint images are registered,
the maximum global rotation between two images cannot be a large value.

###  4.2 Time Complexity and Computational Requirements

A typical palmprint contains $1000$ minutiae on average; then the original MCC
algorithm needs to do $1000\times 1000=10^{6}$ MCC local similarity
computations. Our statistical observation showed that by applying proposed
registration algorithm, we only need to do $12$ percent of all computations,
i.e. $1.2\times 10^{5}$ local MCC similarity computation. Results in table I
confirm this, and show that the contribution of the registration toward the
matching time reduction is significant. By the proposed system,
${{1000}\over{9}}=111$ palmprint matches per second can be done whereas the
feasible number of matches per second for algorithm in [14] (the fastest
algorithm to the best of our knowledge) is ${{1000}\over{36}}=26$. Note that,
we can double the matching speed by applying proposed left-right palm
detector, i.e. average matching time is $4.5$ milliseconds(222 matches per
seconds).

TABLE I: Average matching time (second) Method | Matching time
---|---
Jian and Feng [11] | $1.097$
Dai and Zhou [8] | $5.160$
Chen and Guo [17] | $0.089$
Tariq et.al(CPU) [18] | $0.031$
Proposed method | 0.021
Capelli and Ferrara [14] (Multithreaded) | $0.038$
Proposed method (Multithreaded) | 0.006
TABLE II: Required memory for a typical palmprint image Method | Required memory
---|---
Jian and Feng [11] | $240KB$
Dai and Zhou [8] | $4084KB$
Chen and Guo [17] | less than 8 KB
Tariq et.al [18] | $18KB$
Capelli and Ferrara [14] | $161KB$
Proposed algorithm | $45KB$
TABLE III: EER and FMMR at give FMRs Method | EER | FNMR at FMR$\leq.01\%$ | FNMR at FMR$\leq.001\%$ | FNMR at FMR $=0$
---|---|---|---|---
Jian and Feng [11] | $5.04\%$ | $17.32\%$ | $19.43\%$ | $22.5\%$
Dai and Zhou [8] | $2.99\%$ | $8.78\%$ | $10.45\%$ | $11.58\%$
Wang and Ram [16] | $1.77\%$ | $>10\%$ | $>10\%$ | $>10\%$
Chen and Guo [17] | $0.29\%$ | $2\%$ | $5\%$ | $>10\%$
Tariq et.al [18] | $0.38\%$ | $>0.38\%$ | $>0.38\%$ | $>0.38\%$
Capeli and Ferara [14] | 0.01% | 0.1% | $0.18\%$ | $0.48\%$
Proposed method | $0.04\%$ | $0.12\%$ | 0.14% | 0.24%

The proposed algorithm was tested on an Intel(R) Core(TM) 2 Quad Q9550 CPU at
2.83 GHz, using single and multi-threaded C++ implementation. Table I
summarizes average matching times of the proposed system with those of four
other approaches in [11], [8], [17],[18] and [14]. The times of the methods in
[11], [8] have been measured on an Intel Xeon E5620 at 2.4 GHz, with a single-
thread in C++, and the time of [17] has been measured by Intel(R) core (TM) i7
CPU 2.8 GHz in C++. Time of the algorithm in [14] has been measured by Intel
Core 2 Quad Q9400 CPU at 2.66 GHz, using a multi-threaded C# implementation.
Due to different hardware and the way that software is implemented, it is hard
to accurately compare the matching times of the algorithms. For example, we
implemented the method in [14] in our system using a multi-threaded C++
implementation; and the matching process took about $200$ milliseconds, while
the authors of this algorithm report a matching time of $38$ milliseconds with
a weaker computer and C# implementation.

The computational cost of the registration procedure is composed of applying
CNN,voting and searching for the best transformation parameters. The execution
time of the registration is about four seconds in our experiment. Since it is
performed in the enrollment stage only once, it has no computation complexity
during identification.

The amount of memory which is required by the proposed algorithm to store the
palmprint templates is relatively small in comparison to other algorithms. For
a single minutia, the proposed system stores $8\times 8\times 5$ bits for the
MCC descriptor, and about five bytes for ($x_{m},y_{m},\theta_{m}$). In total
it needs to store $40+5=45$ bytes per minutia which is a small value in
comparison to the $161$ bytes (per minutia) required by the algorithm in [14].
The method in [12] requires $230$ bytes per minutia and one byte per
orientation element; the algorithm in [9] needs five bytes per minutia, two
bytes per orientation. For a typical palmprint ($2040\times 2040$) with an
average $1000$ minutiae and $127\times 127$ orientation elements, the amount
of memory required by these algorithms is summarized in table II. Among all
these algorithms, the proposed system requires relatively less amount of
memory to store minutia descriptors and templates. The method in [17] needs
less memory in comparison to ours, but the matching time and the accuracy of
this algorithm are much worse.

###  4.3  Accuracy

To evaluate the matching algorithms, impostor and genuine tests need to be
done. For this database, the total number of genuine matches is $({{8\times
7}\over{2}}\times 120=3,360)$. Each impression is compared with the remaining
impressions of the same palmprint. The number of impostor matches is much
larger than that of genuine matches. All $8$ impressions from $120$ subjects
are compared with the remaining impressions; therefore, the number of impostor
matches is $({{952\times 960}\over{2}}=456,906)$.

The accuracy of the proposed system is compared to the some state-of-the art(
[11], [8], [14],[16] and [17]). As seen in Table III, our algorithm shows a
higher accuracy at any FMR. At zero FMR, the proposed algorithm makes $8$
false non-match errors. Most of these errors occur because of the images with
low quality, regions with small area and less pattern (similar to latent
palmprints), and less overlap area with other palmprints. When two palmprint
images have no minutia in common, i.e. no overlap, non of the existing
algorithms can find a high matching score between them. It is noteworthy that
without registration , the accuracy of the proposed system reduces
dramatically. This is because of the small values of $n_{R}$, $N_{S}$ and some
other parameters that we have changed in MCC algorithm.

As for the accuracy of the registration phase, $5$ palmprints were not
successfully registered, which is less than one percent of all the palmprints
within the test database. All of the failure cases are due to bad image
quality or pattern-less and small areas. In all of these $5$ images, one of
$M_{s},q_{1}$ or $q_{2}$ was less than the corresponding threshold.

##  5 Conclusion

Because of the existence of creases, a large number of minutia and skin
distortion, minutia-based palmprint matching is a challenging task. In this
paper, we designed new strategy to speed up the palmprint matching speed.
Combining CNN and GHT, an accurate palmprint registration was designed which
can bring all palmprints to a same coordinate system. In the first round of
registration, CNN finds the rotation $\theta$ of a palmprint image respect to
$y$ axis. Image is rotated by $(-\theta)$, and exact registration is done by
GHT. Registration helps to compare only minutiae belonging to the same part of
the palmprint, ignoring comparison of minutiae which are far apart. This
reduces computations in local match phase, and makes the matching algorithm
faster.

Our method is six times faster than the state-of-the-art while the matching
accuracy is still very high.The proposed algorithm was applied on THUPALMLAB
database and the results revealed that our algorithm has the capability of
doing $166$ palmprint matches per second with $EER=0.04\%$.

## References

  * [1] Z. Guo, D. Zhang, L. Zhang, and W. Zuo, “Palmprint verification using binary orientation co-occurrence vector,” _Pattern Recognition Letters_ , vol. 30, pp. 1219–1227, 2009.
  * [2] D. Maltoni, D. Maio, A. Jain, and S. Prabhakar, _Handbook of Fingerprint Recognition_ , 2nd ed. London: Springer-Verlag, 2009.
  * [3] A. Kong, D. Zhang, and M. Kamel, “A survey of palmprint recognition,” _Pattern Recognition_ , vol. 42, pp. 1408–1418, 2009.
  * [4] D. Zhang, W. Zuo, and F. Yue, “A comparative study of palmprint recognition algorithms,” _ACM Computing Surveys_ , vol. 44, p. 2, 2012.
  * [5] D. S. Huang, W. Jia, and D. Zhang, “Palmprint verification based on principal lines,” _Pattern Recognition_ , vol. 41, pp. 1316–1328, 2008.
  * [6] W. Jia, D. S. Huang, and D. Zhang, “Palmprint verification based on robust line orientation code,” _Pattern Recognition_ , vol. 41, pp. 1504–1513, 2008\.
  * [7] X. Wu and Q. Zhao, “Deformed palmprint matching based on stable regions,” _IEEE Transactions on Image Processing_ , vol. 24, pp. 4978–4989, 2015.
  * [8] J. Dai and J. Zhou, “Multifeature-based high-resolution palmprint recognition,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 33, pp. 945–957, 2011.
  * [9] K. C. Mangold, _Data Format for the Interchange of Fingerprint, Facial & Other Biometric Information_, 3rd ed. Gaithersburg, Maryland, USA: ANSI/NIST-ITL, 2016.
  * [10] D. R. Ashbaugh, _Quantitative-Qualitative Friction Ridge Analysis: An Introduction to Basic and Advanced Ridgeology_ , 1st ed. Boca Raton: CRC Press, 1999.
  * [11] A. K. Jain and J. Feng, “Latent palmprint matching,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 31, pp. 1032–1047, 2008.
  * [12] J. Li and G. Shi, “A novel palmprint feature processing method based on skeleton image,” _IEEE International Conference on Signal Image Technology and Internet Based Systems_ , 2008.
  * [13] E. Liu, A. K. Jain, and J. Tian, “A coarse to fine minutiae-based latent palmprint matching,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 35, pp. 2307–2322, 2013.
  * [14] R. Cappelli, M. Ferrara, and D. Maio, “A fast and accurate palmprint recognition system based on minutiae,” _IEEE Trans Syst Man Cybern B Cybern_ , vol. 42, pp. 956–962, 2012.
  * [15] ——, “Minutia cylinder-code: A new representation and matching technique for fingerprint recognition,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 32, pp. 2128–2141, 2010.
  * [16] R. Wang, D. Ramos, R. Veldhuis, J. Fierrez, L. Spreeuwers, and H. Xu, “Regional fusion for high-resolution palmprint recognition using spectral minutiae representation,” _IET Biometrics_ , vol. 3, pp. 94–100, 2014.
  * [17] J. Chen and Z. Guo, “Palmprint matching by minutiae and ridge distance,” _International Conference on Cloud Computing and Security_ , vol. 10040, pp. 371–382, 2016.
  * [18] S. A. Tariq, S. Iqbal, M. Ghafoor, I. A. Taj, N. M. Jafri, S. Razzaq, and T. Zia, “Massively parallel palmprint identification system using gpu,” _Cluster Computing_ , pp. 1–16, 2017.
  * [19] J. Dai, J. Feng, and J. Zhou, “Robust and efficient ridge-based palmprint matching,” _IEEE Trans Pattern Anal Mach Intell_ , vol. 34, pp. 1618–1632, 2012.
  * [20] H. Soleimani and M. Ahmadi, “Fast and efficient minutia-based palmprint matching,” _IET Biometrics_ , vol. http://digital-library. theiet.org/content/journals/10.1049/iet-bmt.2017.0128, 2018.
  * [21] Dai, “Thupalmlab database,” _http://ivg.au.tsinghua.edu.cn/dataset/THUPALM LAB.php_ , 2012.
  * [22] Y. Wang, J. Hu, and D. Phillips, “A fingerprint orientation model based on 2d fourier expansion (fomfe) and its application to singular-point detection and fingerprint indexing,” _IEEE Trans Pattern Anal Mach Intell_ , vol. 29, pp. 573–585, 2007.
  * [23] J. Funada, N. Ohta, M. Mizoguchi, T. Temma, K. Nakanishi, A. Murai, T. Sugiuchi, T. Wakabayashi, and Y. Yamada, “Feature extraction method for palmprint considering elimination of creases,” _Fourteenth International Conference on Pattern Recognition_ , pp. 1051–4651, 1998.
  * [24] C. C. Han, “A hand-based personal authentication using a coarse-to-ﬁne strategy,” _Image and Vision Computing_ , vol. 22, pp. 909–918, 2004.
  * [25] S. Loffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” _32nd International Conference on International Conference on Machine Learning_ , vol. 37, pp. 448–456, 2015.
  * [26] D. A. Clevert, T. Unterthiner, and S. Hochreiter, “Fast and accurate deep network learning by exponential linear units (elus),” _Proceedings of the 4th International Conference on Learning Representation_ , pp. 1–14, 2016\.
  * [27] D. H. Ballard, “Generalizing the hough transform to detect arbitrary shapes,” _Pattern Recognition_ , vol. 13, pp. 111–122, 1981.
  * [28] L. Hong, Y. Wan, and A. Jain, “Fingerprint image enhancement: algorithm and performance evaluation,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 20, pp. 777–789, 1998.
  * [29] T. Y. Zhang and C. Y. Suen, “A fast parallel algorithm for thinning digital patterns,” _Communications of the ACM_ , vol. 27, pp. 236–239, 1984.
  * [30] F. Zhao and X. Tang, “Preprocessing and postprocessing for skeleton-based fingerprint minutiae extraction,” _Pattern Recognition_ , vol. 40, pp. 1270–1281, 2007.