title: Python defaultdict for large data sets
body: <p>I am using <code>defaultdict</code> to store millions of phrases, so my data structure looks like <code>mydict['string'] = set(['other', 'strings'])</code>. It seems to work ok for smaller sets but when I hit anything over 10 million keys, my program just crashes with the helpful message of <code>Process killed</code>. I know <code>defaultdict</code>s are memory heavy, but is there an optimised method of storing using <code>defaultdict</code>s or would I have to look at other data structures like numpy array?</p>
  
  <p>Thank you </p>