title: marshal dumps faster, cPickle loads faster
body: <p>I'm implementing a program that needs to serialize and deserialize large objects, so I was making some tests with <code>pickle</code>, <code>cPickle</code> and <code>marshal</code> modules to choose the best module. Along the way I found something very interesting:</p>
  
  <p>I'm using <code>dumps</code> and then <code>loads</code> (for each module) on a list of dicts, tuples, ints, float and strings.</p>
  
  <p>This is the output of my benchmark:</p>
  
  <pre><code>DUMPING a list of length 7340032
  ----------------------------------------------------------------------
  pickle =&gt; 14.675 seconds
  length of pickle serialized string: 31457430
  
  cPickle =&gt; 2.619 seconds
  length of cPickle serialized string: 31457457
  
  marshal =&gt; 0.991 seconds
  length of marshal serialized string: 117440540
  
  LOADING a list of length: 7340032
  ----------------------------------------------------------------------
  pickle =&gt; 13.768 seconds
  (same length?) 7340032 == 7340032
  
  cPickle =&gt; 2.038 seconds
  (same length?) 7340032 == 7340032
  
  marshal =&gt; 6.378 seconds
  (same length?) 7340032 == 7340032
  </code></pre>
  
  <p>So, from these results we can see that <code>marshal</code> was extremely fast in the <strong>dumping</strong> part of the benchmark:</p>
  
  <blockquote>
    <p>14.8x times faster than <code>pickle</code> and 2.6x times faster than <code>cPickle</code>.</p>
  </blockquote>
  
  <p>But, for my big surprise, <code>marshal</code> was by far slower than <code>cPickle</code> in the <strong>loading</strong> part:</p>
  
  <blockquote>
    <p>2.2x times faster than <code>pickle</code>, but 3.1x times slower than <code>cPickle</code>.</p>
  </blockquote>
  
  <p>And as for RAM, <code>marshal</code> performance while <strong>loading</strong> was also very inefficient:</p>
  
  <p><img src="https://i.stack.imgur.com/ZAFoV.png" alt="Ubuntu System Monitor"></p>
  
  <p>I'm guessing the reason why loading with <code>marshal</code> is so slow is somehow related with the length of the its serialized string (much longer than <code>pickle</code> and <code>cPickle</code>).</p>
  
  <ul>
  <li>Why <code>marshal</code> dumps faster and loads slower?</li>
  <li>Why <code>marshal</code> serialized string is so long?</li>
  <li>Why <code>marshal</code>'s loading is so inefficient in RAM?</li>
  <li>Is there a way to improve <code>marshal</code>'s loading performance?</li>
  <li>Is there a way to merge <code>marshal</code> fast dumping with <code>cPickle</code> fast loading?</li>
  </ul>