Whats the best way to sort a dataset into groups by using a facial recognition algorithm that compares only 2 images at a time? I have a facial recognition algorithm that compares two images A and B and returns the likelihood that they match.

I also have 50,000 images, and I would like to sort these images into groups.

Here's the immediate way I thought of to do this:


Start with image 0, compare it to all 49,999 images. Store the similarities in an array
Move on to image 1, compare it to 49,998 images (skip img_0). Store the similarities in an array


At the end of it all I'm left with a verified_listof images that matched, and I can basically feed them into a network to combine them, ie if I have a verified_list like so:

[[1,2,3],[2,5,6],[8,9]] (verified_list is the same length as the number of images, so verified_list[1] contains references all the images that image_1 matches with)

then the network graph combines them into:

[[1,2,3,5,6], [8,9]

Indicating I have two groups.

Obviously this is a huge amount of processing, I think it equates to: nCr(50000,2), some huge number!

Is there a faster way to sort my dataset into groups?
So it appears you actually have vectors over $\mathbb{R}^n$, one vector per face, and you want to group the vectors, using $L_2$ distance (Euclidean distance) to measure similarity.

There are a number of techniques for this.  See data structures for nearest neighbor search (https://en.wikipedia.org/wiki/Nearest_neighbor_search).  I would suggest that locality sensitive hashing might be a promising choice for your situation.  You could also look into clustering algorithms (though they will probably still require nearest neighbor search as one step of their working).