title: Best way to compare two large sets of strings in Python
body: <p>I am using Python (and have access to pandas, numpy, scipy).</p>
  
  <p>I have two sets strings set A and set B. Each set A and B contains c. 2000 elements (each element being a string). The strings are around 50-100 characters long comprising up to c. 20 words (these sets may get much larger).</p>
  
  <p>I wish to check if an member of set A is also a member of set B.</p>
  
  <p>Now I am thinking a naive implementation can be visualised as a matrix where members in A and B are compared to one another (e.g. A1 == B1, A1 == B2, A1 == B3 and so on...) and the booleans (0, 1) from the comparison comprise the elements of the matrix.</p>
  
  <p>What is the best way to implement this efficiently?</p>
  
  <p>Two further elaborations:</p>
  
  <p>(i) I am also thinking that for larger sets I may use a Bloom Filter (e.g. using PyBloom, pybloomfilter) to hash each string (i.e. I dont mind fasle positives so much...). Is this a good approach or are there other strategies I should consider?</p>
  
  <p>(ii) I am thinking of including a Levenshtein distance match between strings (which I know can be slow) as I may need fuzzy matches - is there a way of combining this with the approach in (i) or otherwise making it more efficient?</p>
  
  <p>Thanks in advance for any help!</p>