title: Finding matches between 30,000+ data sets
body: <p>For simplicity sake, lets use a case example of 3 colors with corresponding numbers (there is actually 30,000+ different 'colors' and 254 different 'numbers' in real life though)</p>
  
  <p>Red   - 0, 1, 2, 3, 10, 15</p>
  
  <p>Green - 0, 2, 3, 20</p>
  
  <p>Blue  - 2, 10, 11, 12</p>
  
  <p>I want to find the matches between them (rgb, rg, rb), as well as keep a tally of the number of #s shared between the set:</p>
  
  <p>rgb = 1</p>
  
  <p>rg = 2</p>
  
  <p>rb = 2</p>
  
  <p>Finally it'll need to determine a ratio of the number of #s shared compared to the number of distinct #s in the set.</p>
  
  <p>rgb = 1/9 (since it has a distinct: 0, 1, 2, 3, 10, 11, 12, 15, 20)</p>
  
  <p>rg = 2/7 (0, 1, 2, 3, 10, 15, 20)</p>
  
  <p>rb = 2/8 (0, 1, 2, 3, 10, 11, 12, 15)</p>
  
  <hr>
  
  <p>So the total output would be</p>
  
  <p>match | # of matches | % |</p>
  
  <p>rgb   |    1  |  1/9</p>
  
  <p>rg    |  2   |   2/7</p>
  
  <p>rb     | 2   |  2/8</p>
  
  <hr>
  
  <p>The algorithm I was able to come up w/is you have each color in a table and map the numbers associated with it (aka red (table Name), 0, 1, 2, 3, 10, 15 (data)). Then take the color with the most 'numbers' and compare it to every other color's hours, find matches.  Once done with that color you can 100% ignore it and move onto the next color and do comparisons with n-1.</p>
  
  <p>Take the example:</p>
  
  <p>1) Select red</p>
  
  <p>2) Does any other color share 0</p>
  
  <p>3) Does any other color share 1</p>
  
  <p>....etc</p>
  
  <p>4) Select blue
  5) Does any other color minus red share .....</p>
  
  <p>I know that there has to be a more efficient way to do this, any suggestions?</p>
  
  <p>Thanks for the help.</p>