title: How to intersect two sets of longs that don't fit into memory?
body: <p>I've got an interesting problem. I want to intersect two Sets of Longs in Java, with each set having 1B members - this is 4GB per set. This will not fit into memory on the server that I need to run it on.</p>
  
  <p>I'm wondering what interesting methods are out there for solving this problem.</p>
  
  <p>What I've come up with so far is reading subsets of each original set from disk that are small enough to fit into memory, then intersecting each subset, and writing those to disk temporarily. Finally, I could go through and intersect these subsets. I get a feeling that this may turn into a map reduce job.</p>
  
  <p>Maybe you'll have some better ideas :) I doubt I'm the first person to have come up with this problem.</p>