title: How can I make a large python data structure more efficient to unpickle?
body: <p>I have a list of ~1.7 million "token" objects, along with a list of ~130,000 "structure" objects which reference the token objects and group them into, well, structures. It's an ~800MB memory footprint, on a good day.</p>
  
  <p>I'm using <code>__slots__</code> to keep my memory footprint down, so my <code>__getstate__</code> returns a tuple of serializable values, which <code>__setstate__</code> bungs back into place. I'm also not pickling all the instance data, just 5 items for tokens, 7-9 items for structures, all strings or integers.</p>
  
  <p>Of course, I'm using cPickle, and HIGHEST_PROTOCOL, which happens to be 2 (python 2.6). The resulting pickle file is ~120MB.</p>
  
  <p>On my development machine, it takes ~2 minutes to unpickle the pickle. I'd like to make this faster. What methods might be available to me, beyond faster hardware and what I'm already doing?</p>