1. A biometric authentication system comprising: a data storage system configured to maintain, for a group of multiple, different people, biometric data that includes sorted similarity scores, wherein each similarity score represents a similarity between a biometric image of at least a portion of a corresponding person from the group of multiple, different people and a reference image; and a biometric verification system that includes at least one processor, the biometric verification system being configured to: access data for a particular biometric image of at least a portion of a particular person; compute a particular similarity score that represents similarity between the particular biometric image and the reference image used in computing the similarity scores for each person in the group of multiple, different people; search, using the computed particular similarity score, the sorted similarity scores included in the biometric data for the group of multiple, different people to find a closest match to the computed particular similarity score in the sorted similarity scores; compute a difference between the computed particular similarity score and a similarity score of the closest match; determine whether the difference satisfies a threshold difference value; and output the closest match based on a determination that the difference satisfies the threshold difference value.