title: Pandas msgpack vs pickle
body: <p><code>msgpack</code> in Pandas is supposed to be a replacement for <code>pickle</code>.</p>
  
  <p>Per the <a href="http://pandas.pydata.org/pandas-docs/stable/io.html#msgpack-experimental" rel="noreferrer">Pandas docs on msgpack</a>:</p>
  
  <blockquote>
    <p>This is a lightweight portable binary format, similar to binary JSON,
    that is highly space efficient, and provides good performance both on
    the writing (serialization), and reading (deserialization).</p>
  </blockquote>
  
  <p>I find, however, that its performance does not appear to stack up against pickle.</p>
  
  <pre><code>df = pd.DataFrame(np.random.randn(10000, 100))
  
  &gt;&gt;&gt; %timeit df.to_pickle('test.p')
  10 loops, best of 3: 22.4 ms per loop
  
  &gt;&gt;&gt; %timeit df.to_msgpack('test.msg')
  10 loops, best of 3: 36.4 ms per loop
  
  &gt;&gt;&gt; %timeit pd.read_pickle('test.p')
  100 loops, best of 3: 10.5 ms per loop
  
  &gt;&gt;&gt; %timeit pd.read_msgpack('test.msg')
  10 loops, best of 3: 24.6 ms per loop
  </code></pre>
  
  <p><strong>Question:</strong>  Asides from potential security issues with pickle, what are the benefits of msgpack over pickle?  Is pickle still the preferred method of serializing data, or do better alternatives currently exist?</p>