1. A biometric authentication system comprising: a database retaining an enrollee ID for each of N enrollees, registered templates of biometric information of at least one kind for each of N enrollees, and a score table for recording scores, each of the scores representing a similarity between a registered template and a corresponding registered template of another enrollee; a prior probability setting function for setting prior probabilities that an authentication object user will be same person as the respective enrollees; biometric information input sensors for acquiring biometric information of at least one kind from the authentication object user; a feature extraction function for extracting a feature from the acquired biometric information; a 1:N fast matching function for matching the feature of the authentication object user with the registered templates of the enrollees numbering a predetermined threshold K (K<N), wherein the 1:N fast matching function rearranges a matching sequence of the registered templates by referring to the score table; a delta score calculation function for calculating a delta score which represents how much two scores differ from each other, by using a score obtained for each of the registered templates by the 1:N fast matching and using the score table; a posterior probability calculation function for calculating posterior probabilities that the authentication object user will be same person as the respective enrollees, on the basis of the score and the delta score; and an authentication object user identification function for conducting identification processing of the authentication object user by comparing the posterior probabilities with a predetermined threshold A.