I've written a python program that is responsible for making sure that all elements in one data set (let's call it the source) exists in another (let's call it the target).Constraints beyond my control:Retrieving unique element identifiers from the source is extremely costly (~7-8 minutes), whereas counting the number of elements is feasible (~900ms).The unique identifiers are sortable but non-linear: It's not possible to find missing elements by looking at the identifiers alone.Missing or extra elements could be randomly distributed in the target, so I can't assume that they would be grouped in the beginning or end when sorting by the unique identifier.Given the above constraints, I wrote a method that uses binary search to find and return the first mismatch between source and target. I suspect this is an obscure, non-canonical solution to a fairly trivial problem but I have no developer to get feedback from, so I'd like to have the following aspects reviewed:Have I approached the problem correctly?If so, does variable naming and comments convey the problem and solution in a way that makes it easy to understand for an outside developer?My solution won't detect mismatch in a range if it contains an equal number of extra and missing elements. Is there a way to accommodate that flaw?The method:def first_mismatch(self, threshold):          Finds the first mismatch between source and target     and returns the position and mismatch type in the format     ((first_sourceid, last_sourceid), extra).     first/last sourceid represents the range where the mismatch is     located and extra is True if the number of elements in target is      greater than in source.     the threshold argument represents the least acceptable distance     between first/last sourceid before exiting the binary search.     Setting threshold to 1 would home in the mismatch with perfect      precision but will require an unnecessarily high number of      count() requests against the source.          min_index = 0     max_index = len(self.target) - 1     if max_index < 0:         return ((None, None), False)     # set arguments for diff to strictly count elements     diff_args = {         'field': self.model.COUNTER_FIELD,         'by_queryset': False,         'count': True,          'exclude_empty': False,         }     # Upper edge case: Elements match up to last target element     sourceid_range = (         None,         self.sourceid_from_target_instance(self.target[max_index]),         )     target_count, source_count = self.diff(         sourceid_range=sourceid_range, **diff_args         )     if target_count == source_count:         return ((sourceid_range[1], None), False)      # Lower edge case: Element mismatch before first target element     sourceid_range = (         None,         self.sourceid_from_target_instance(self.target[min_index]),         )     target_count, source_count = self.diff(         sourceid_range=sourceid_range, **diff_args)     if target_count != source_count:         return ((None, sourceid_range[1]), target_count > source_count)      while True:         mid_index = min_index + (max_index - min_index)/2         sourceid_range = (           self.sourceid_from_target_instance(self.target[min_index]),           self.sourceid_from_target_instance(self.target[mid_index]),           )         target_count, source_count = self.diff(             sourceid_range=sourceid_range, **diff_args)         diff = target_count != source_count         min_index, max_index = (             (min_index, mid_index)             if diff else             (mid_index, max_index)             )         if diff and (max_index - min_index <= threshold):             return (         (self.sourceid_from_target_instance(self.target[min_index]),          self.sourceid_from_target_instance(self.target[max_index])          ),          target_count > source_count,         )The method is a part of a class Sync located in a module models.py (part of django): https://ideone.com/pH44zo. The method is defined at row# 176 and called at row# 122.
Locate missing or extra elements between two data sets with limited access to unique identifiers
python