What is the efficient way to generate a similarity score when comparing two face images? I am working on a face recognition application using deep learning. To plot the ROC curves and do performance evaluation, I extracted the features from the last layer of the deep neural network and I have tried Euclidian and Manhattan distances between high dimensional feature representation, but this won't give me accurate measure in terms of comparison as faces have intra-personal and interpersonal differences and other challenges such as pose, lighting condition, and expression. I came across techniques such as joint bayesian, sparse projection, KL-divergence based regularizer/prior(EM algorithm) etc. Are there any other approaches that I'm missing here? What would be the efficient approach to generate similarity scores?
A common method is defining "landmarks", a set of specific points that exist on every face . For example, the outside edge of each eye or the bottom of the chin are landmarks on most faces. Train the deep learning network to learn the relationship between these landmarks which will form the embedding space. Then define a distance metric in this embedding space. Finally, create a threshold for similarity between sets of points in this embedding space. That threshold can be learned through "one-class" classification techniques.

One gotcha is pose. Faces turned different directions look totally different to a computer. Assuming you have isolated the faces in an image, it is best practice to warp each face so that the eyes and lips are always in the place in the image. This (and other normalization steps) should be done early in the processing pipeline to make subsequent analysis much easier.