finger vein recognition is a promising biometric recognition technology which verifies identities through finger vein patterns . medical studies have shown that the finger vein pattern is unique and stable . in detail , the finger veins of an individual are different from the others ' , and even the veins captured from a single individual are quite different from one finger to another . furthermore , the finger veins are also invariant for healthy adults . compared with fingerprints , finger veins are hard to be forged or stolen as they are hidden inside the fingers . the contactless captures of finger veins also ensure both convenience and cleanliness , and they are user - friendly . furthermore , finger veins are less affected by physiology and environment factors such as dry skin and dirt . thirdly , the features are extracted . finally , the finger vein images are matched based on the extracted features . the preprocessing procedure includes image enhancement , normalization , and segmentation . for image enhancement , yang and yan incorporated directional decomposition and frangi filtering to enhance the image quality . yang and yang introduced multi - channel gabor filter to enhance the images and obtained better performance . finger vein segmentation is also a very important step , and there are some typical methods including line tracking , mean curvatures , and region growth - based feature . however , a summary of these approaches with the typical references is provided in table 1 . wu and liu extracted the pca features and then trained a neural network for matching , which results in a high recognition rate . since pca transforms the 2-dimensional image matrix to a 1-dimensional vector , the covariance matrix is always large in size and it is time intensive to obtain the projection matrix which is composed of the covariance matrix 's eigenvectors . yang et al . proposed 2dpca to reduce the size of the covariance matrix and save time for computing projection matrices . in order to represent the characteristics of the 2-dimensional images more accurately , zhang and zhou introduced ( 2d ) pca which can reflect the information of the image in row and column directions , respectively , use less time to compute the projection matrix , and get better experimental results on face recognition wu and liu used pca and lda to extract features and train a svm model for recognition . measuring for example , knn requires a distance metric to find the neighbors of the target instance and then conducts classification or regression based on the distance metric . typical distance metrics , such as euclidean distance , make significant contribution in some application domains . in some conditions , these metrics can not satisfy the assumption that the distances between instances from the same class are small while those from different classes are large . there are two challenges for finger vein recognition : ( 1 ) how to efficiently extract distinguishing features and ( 2 ) how to design a strong classifier with high recognition rate and fast recognition speed to make the system more practical in real - world applications . to overcome these two challenges , in this paper we apply ( 2d ) pca to extract the features from finger vein images . in order to address the shortcoming of traditional distance - metric - based classifiers , we build a classifier for each individual based on metric learning . with regard to training samples of each classifier , thus , we use smote technology to oversample the positive samples to balance the two classes before training the classifier . the experimental results show that the proposed method has good performance on finger vein recognition . pca is a typical linear dimensionality reduction and feature extraction method . due to the transformation from the 2-dimensional image matrix into a 1-dimensional column vector , pca often makes the size of the corresponding covariance matrix too large , and computing the eigenvectors and eigenvalues becomes time - consuming . in order to solve this problem , yang et al . 2dpca directly uses the image matrix to compute pca features without transforming the 2-dimensional image matrix into a 1-dimensional column vector . therefore , it reduces the size of corresponding covariance matrix and obtains the feature projection matrix with less time . however , 2dpca works only for the row direction of images . to address the problem , zhang and zhou proposed ( 2d ) pca which captures the image information from not only the row direction but also the column direction . the experimental results show that ( 2d ) pca outperforms 2dpca and pca in terms of both recognition rate and running time . the process of ( 2d ) pca is described as follows . considering m finger vein images , which are denoted by a1 ,  , am , we compute the mean image matrix as a = ( 1/m)jaj and the image covariance matrix gas 
 ( 1)g=  1mj=1m(aja)t(aja ) . 
  for a random image matrix a , the key of obtaining the new features is to get the projection matrix x  r , nd . then the new features are calculated as y = ax . the total scatter of the projected samples is used to determine a good projection matrix x , where the total scatter of the projected samples can be characterized by the trace of the covariance matrix of the projected feature vectors . from this point of view , we adopt the following criterion : 
 ( 2 )  j(x)=trace{e[(ye(y))(ye(y))t ] }  = trace{e[(ax  e(ax))(ax  e(ax))t ] }  = trace{xte[(ae(a))t(ae(a))]x}. 
 
 ( 3)j(x)=trace{xtgx}. 
  it has been proven that j(x ) gets the maximum when the projection matrix x is composed by the d orthonormal eigenvectors coupled to the d largest eigenvalues . in so saying , x obtains the optional value , and d can be controlled by setting a threshold as follows : 
 ( 4)i=1dij=1nj , 
 where  is a user - specific threshold and 1 , 2 ,  , n is the top - n largest eigenvalues of g. because x only reflects the information in the row direction , zhang and zhou proposed alternative 2dpca which reflects the information in the column direction and combines 2dpca with alternative 2dpca to obtain a new method called ( 2d ) pca . here is the process of alternative 2dpca . let the image matrix aj = [ ( aj),  ,(aj ) ] , and the mean image matrix a=[(a(1))t,  ,(a(n))t]t , where aj and a(i ) , denote the ith row vector of aj and a(i ) respectively . the image covariance matrix can be rewritten as 
 ( 5)g1=1mj=1 m  k=1m(aj(k)a(k))t(aj(k)a(k ) ) . 
  similarly , to achieve the projected matrix x in 2dpca , we can obtain the projection matrix z  r from ( 2 ) and ( 5 ) . we can also compute q in the same manner as we compute d in 2dpca . using the projected matrix x , z in 2dpca and alternative 2dpca , respectively , we can obtain the new feature we can see from ( 6 ) that the new feature c reflects more information of the image than the features obtained by 2dpca and alternative 2dpca . furthermore , the dimension of c is smaller , and thus ( 2d ) pca costs less time than 2dpca and alternative 2dpca for image processing . the main task of the metric learning is to find a better distance metric , based on which the distances between the samples from same class become small while those from different classes become large . this helps to improve the performance of the machine learning methods . to overcome the shortage of the knn classifier using euclidean distance , weinberge et al . proposed a metric learning method called lmnn ( large margin nearest neighbor )   which learns a distance metric to improve the performance of knn classifiers . the metric is obtained by learning a linear transformation matrix l. with this distance metric , the distance between the same - class instances becomes smaller , and they are separated from the other instances by a large margin . ( i = 1 ,  , n ) denote the feature vector of training instances and the essence of lmnn is to obtain a new distance d(xi , xj ) = ||l(xixj)|| = ( xixj)ll(xi  xj ) after learning a linear transformation l matrix . with this distance metric , the distance between the instance and its k nearest neighbors will be minimized and the distance between the instances in different classes will be larger . green circles denote instances from the first class , yellow squares denote instances from the second class , and red squares denote instances from the third class . consider the instance denoted by the white circle , which is treated as a test instance from the first class , in our following analysis . based on euclidean distance , we find 4 nearest neighbors , and this test instance is misclassified into the second class . however , using the lmnn - learned metric ; this instance is separated from the second and the third instances . the other is to resample the original dataset , either by oversampling the minority class and/or undersampling the majority class . chawla et al . proposed an oversampling approach called smote where the minority class is oversampled by creating the minority class is oversampled by taking each minority class sample and introducing synthetic examples along the line segments joining any / all of the k minority class nearest neighbors . depending upon the amount of oversampling required , neighbors from the k nearest neighbors are randomly chosen . the proposed method includes training process and recognition process . as shown in figure 2 , a classifier is built for each individual , and the samples from a certain individual are treated as positive and others are negative . in the verification mode , we input a test sample to corresponding classifier to verify whether the sample comes from this individual based on the classification result . in the identification mode , we input a test sample to every classifier and identify which individual this sample belongs to . in the training process , it is necessary to preprocess the infrared images of the finger veins . preprocessing includes grayscale , roi selection , and normalization ( e.g. , size normalization and gray normalization ) . after the preprocessing , we apply ( 2d ) pca to extract the features of the training samples . then we label the samples as positive and negative class accordingly and oversample the positive samples with smote . we learn a new distance metric , that is , the transformation matrix l , with lmnn . the preprocessing and feature extraction in the recognition process are similar to that in the training process . after that , we input the features of the samples to train classifier to verify the individual based on the classification result . the preprocessing includes image grayscale , roi selection , size normalization , and gray normalization . the original image ( an example is shown in figure 3(a ) ) is a 24-bit color image with a size of 320  240 . in order to reduce the computational complexity , we transform the original image to an 8-bit image based on the gray - scale equation y = r    0.299 + g    0.588 + b    0.114 , where r , g , and b denote the value of red , green , and blue . as the background of finger vein region might include noise , we employ an edge - detection method to segment the finger vein region from the gray - scale image . a sobel operator with a 3  3 mask [ -101 - 202 - 101 ] is used for detecting the edges of fingers . the width of the finger region can be obtained based on the maximum and minimum abscissa values of the finger profile , and the height of the finger region is similarly detected . the size of the selected roi is different from image to image due to personal factors such as different finger size and changing location . therefore it is necessary to normalize the roi region to the same size before the feature extraction process by ( 2d ) pca . we use bilinear interpolation for size normalization in this paper , and the size of the normalized roi is set to be 96    64 ( as shown in figure 3(c ) ) . in order to extract efficient features , gray normalization is used to obtain a uniform gray distribution ( as shown in figure 3(d ) ) . formally , we have 
 ( 7)p(i , j)=p(i , j)g1g2g1 , 
 
 where  p(i , j ) is the pixel value of the original image , g1 is the min pixel value of original image , g2 is the max pixel value of original image , and p(i , j ) is the pixel value of image after gray normalization . after the preprocessing , we extract the features for each image by ( 2d ) pca and assign labels for them . a classifier is trained for every individual , where the samples belonging to this individual are treated as positive and others are negative . we oversample the positive samples by smote to obtain an augmented training set which achieves class balance in general . lmnn is then used on the augmented training set to obtain a transformation matrix l. with this new distance metric , a knn classifier is built for classification . in the verification mode , we input the feature vector of a test sample to a classifier which represents a certain individual , and then we verify whether the sample belongs to this individual based on the classification result . in the identification mode , we employ all classifiers to classify the test sample . if only a classifier c classifies it as positive class , this sample belongs to the individual which corresponds to the classifier c. if there are many classifiers classifying the sample as positive class , then we use the training accuracy rate for decision making : the sample belongs to the individual that corresponds to the classifier with the best training accuracy . the experiments were conducted using our finger vein database which is collected from 80 individuals ' ( including 64 males and 16 females , asian race ) index fingers of right hand , where each index finger contributes 18 finger vein images . each individual participated in two sessions , separated by two weeks ( 14 days ) . the age of the participants was between 19 and 60 years , and their occupations included university students , professors , and workers at our school . the capture device was manufactured by the joint lab for intelligent computing and intelligent system of wuhan university , china , which is illustrated in figure 4 . after roi extraction and size normalization , the size of the region used for feature extraction is reduced to 96  64 . therefore , there are 80 classes , where each class contains 18 samples in our database . all the experiments are implemented with matlab and conducted on a machine with 2.4  g cpu and 4  g memory . we design three experiments to verify the efficiency of the proposed method . in experiment 1 , we extract features by ( 2d ) pca and then compare the classification performance of the metric - learning - based method and the classic euclidean - distance - based method . in experiment 2 , we compare the classification performance of knn classifier combined with lmnn using different number of training samples . in experiment 3 we select 480 , 720 , 960 , and 1200 images ( i.e. , 6 , 9 , 12 , and 15 images for each individual ) for training , and the rest of 960 , 720 , 480 , and 240 images ( i.e. , 12 , 9 , 6 , and 3 images for each individual ) are left for testing , respectively . we treat the training samples from each individual as the positive class and construct a center point for each class , where the ith feature of the center point is calculated by averaging the corresponding feature values of the training samples . as there are 80 individuals , we then obtain 80 center point ci  ( i=1,2,  ,80 ) . for any testing sample c , we estimate the euclidean distances from sample c to each center point , d(c , ci)=||c - ci||2  ( i=1,2,  ,80 ) . if ( c , cj)=mind(c , ci)(i = 1,2 ,  , 80 ) , then c goes to the jth class.the metric - learning - based method works similarly as the euclidean - distance - based method except for the usage of the learned distance metric d(c , ci)=||l(c - ci)||2  ( i=1,2,  ,80 ) . it is clearly seen that the recognition rate of the metric - learning - based method is higher than the euclidean - distance - based method . with distance metric transformation , two samples from different classes with small euclidean distance are dragged farther . on the other hand , next we are going to provide an intuitive explanation based on the example shown in figures 6 and 7.these two figures show the data distribution of the data set with 480 training samples and 960 testing samples . we obtain 25 features for each sample using ( 2d ) pca and select 2 features with the largest contribution to euclidean distance metric . similarly , these two features are transformed to a new metric space using lmnn , as shown in figure 7 . the samples of the first individual ( including 6 training images and 12 testing images ) are treated as positive , and the rest of them are considered as negative . in figures 6 and 7 , we use red plus to denote positive training samples , green plus for positive testing samples , blue star for negative training samples , and yellow star for negative testing samples . it is shown from figure 6 that it is difficult to distinguish the first class from the others because the distances between samples in the first class and the other classes are indiscriminating . however , by using lmnn , the samples in the first class are gathered together , as shown in figure 7 . in detail , the positive samples are located mainly in the area of abscissa value between 0 and 10 . on the contrary , this makes it easier to discriminate samples in the first class from samples in the other classes . 
  in this experiment , we select 480 , 720 , 960 , and 1200 images ( i.e. , 6 , 9 , 12 , and 15 images for each individual ) for training , and the rest of 960 , 720 , 480 , and 240 images ( i.e. , 12 , 9 , 6 , and 3 images for each individual ) are left for testing , respectively . we treat the training samples from each individual as the positive class and construct a center point for each class , where the ith feature of the center point is calculated by averaging the corresponding feature values of the training samples . as there are 80 individuals , we then obtain 80 center point ci  ( i=1,2,  ,80 ) . for any testing sample c , we estimate the euclidean distances from sample c to each center point , d(c , ci)=||c - ci||2  ( i=1,2,  ,80 ) . if ( c , cj)=mind(c , ci)(i = 1,2 ,  , 80 ) , then c goes to the jth class . the metric - learning - based method works similarly as the euclidean - distance - based method except for the usage of the learned distance metric d(c , ci)=||l(c - ci)||2  ( i=1,2,  ,80 ) . it is clearly seen that the recognition rate of the metric - learning - based method is higher than the euclidean - distance - based method . with distance metric transformation , two samples from different classes with small euclidean distance are dragged farther . on the other hand , next we are going to provide an intuitive explanation based on the example shown in figures 6 and 7 . these two figures show the data distribution of the data set with 480 training samples and 960 testing samples . we obtain 25 features for each sample using ( 2d ) pca and select 2 features with the largest contribution to euclidean distance metric . similarly , these two features are transformed to a new metric space using lmnn , as shown in figure 7 . the samples of the first individual ( including 6 training images and 12 testing images ) are treated as positive , and the rest of them are considered as negative . in figures 6 and 7 , we use red plus to denote positive training samples , green plus for positive testing samples , blue star for negative training samples , and yellow star for negative testing samples . it is shown from figure 6 that it is difficult to distinguish the first class from the others because the distances between samples in the first class and the other classes are indiscriminating . however , by using lmnn , the samples in the first class are gathered together , as shown in figure 7 . in detail , the positive samples are located mainly in the area of abscissa value between 0 and 10 . on the contrary , this makes it easier to discriminate samples in the first class from samples in the other classes . experiment 2 in this experiment , we select 6 , 9 , 12 , and 15 images from each individual as training samples to build a knn classifier . here the number of neighbors , that is , k , is empirically set to be 3 in knn . we obtain different recognition rates with different numbers of training samples , and the experimental results are shown in figure 8.overall , the recognition rate increases with the number of training images increases . when the number of the training images goes to 15 , the recognition rate reaches 96.67% . it is also worth noting that , as compared to table 2 , the knn - based method outperforms the above - mentioned metric - learning - based method and the euclidean - distance - based method , by considering the same number of training images . 
  in this experiment , we select 6 , 9 , 12 , and 15 images from each individual as training samples to build a knn classifier . the number of neighbors , that is , k , is empirically set to be 3 in knn . we obtain different recognition rates with different numbers of training samples , and the experimental results are shown in figure 8 . when the number of the training images goes to 15 , the recognition rate reaches 96.67% . it is also worth noting that , as compared to table 2 , the knn - based method outperforms the above - mentioned metric - learning - based method and the euclidean - distance - based method , by considering the same number of training images . we select 1200 images ( 15 images for each individual ) for training and 240 images ( 3 images for each individual ) for testing . we use smote to oversample the positive samples to be 5 , 10 , 20 , 30 , 40 , and 50 times as large as the original set . we observe that the recognition rate does not improve by only increasing a small number of synthetic positive samples , as shown in smote-5 and smote-10 . after that , the recognition rate increases by about 3% , and finally it achieves 99.17% with smote-40 or smote-50 . with a sufficiently large set of synthetic positive samples , we select 1200 images ( 15 images for each individual ) for training and 240 images ( 3 images for each individual ) for testing . we use smote to oversample the positive samples to be 5 , 10 , 20 , 30 , 40 , and 50 times as large as the original set . we observe that the recognition rate does not improve by only increasing a small number of synthetic positive samples , as shown in smote-5 and smote-10 . after that , the recognition rate increases by about 3% , and finally it achieves 99.17% with smote-40 or smote-50 . with a sufficiently large set of synthetic positive samples , the recognition performance would not improve any more . this paper proposes a new finger vein recognition method based on ( 2d ) pca and metric learning . firstly , we extract features by ( 2d ) pca and then train a binary classifier for each individual based on metric learning . furthermore , we address the class imbalance problem by using smote oversampling before the classifier is trained . ( 1 ) we apply ( 2d ) pca to extract features of finger vein image , where ( 2d ) pca reflects the information in both the row direction and the column direction , and it is more efficient for feature extraction as compared to pca and 2dpca . ( 2 ) we build the knn classifier based on metric learning using lmnn which changes the sample distribution in the new metric space . lmnn makes the distance between the samples from the same class smaller and the distance between the samples from different classes larger . this is incorporated with individually trained classifiers which reflect the characteristics of the corresponding individuals . ( 3 ) we note the class - imbalance problem ; that is , when building the classifier for an individual , the number of the samples from the other individuals is considerably large . we tackle it by oversampling the positive samples with smote , and the experimental results validate the effectiveness . promising future work includes the exploration of features with better discrimination as well as the processing finger vein images of low quality .
finger vein recognition is a promising biometric recognition technology , which verifies identities via the vein patterns in the fingers . in this paper , ( 
 2d)2 pca is applied to extract features of finger veins , based on which a new recognition method is proposed in conjunction with metric learning . 
 it learns a knn classifier for each individual , which is different from the traditional methods where a fixed threshold is employed for all individuals . besides , the smote technology is adopted to solve the class - imbalance problem . 
 our experiments show that the proposed method is effective by achieving a recognition rate of 99.17% .