title: What are the differences between feather and parquet?
body: <p>Both are <strong>columnar (disk-)storage formats</strong> for use in data analysis systems. 
  Both are integrated within <a href="https://arrow.apache.org/" rel="noreferrer">Apache Arrow</a> (<a href="https://pypi.python.org/pypi/pyarrow" rel="noreferrer">pyarrow</a> package for python) and are
  designed to correspond with <a href="https://arrow.apache.org/docs/python/index.html" rel="noreferrer">Arrow</a> as a columnar in-memory analytics layer.</p>
  
  <p>How do both formats differ?</p>
  
  <p>Should you always prefer feather when working with pandas when possible?</p>
  
  <p>What are the use cases where <a href="https://arrow.apache.org/docs/python/ipc.html#feather-format" rel="noreferrer">feather</a> is more suitable than <a href="https://arrow.apache.org/docs/python/parquet.html" rel="noreferrer">parquet</a> and the 
  other way round?</p>
  
  <hr>
  
  <p>Appendix</p>
  
  <p>I found some hints here <a href="https://github.com/wesm/feather/issues/188" rel="noreferrer">https://github.com/wesm/feather/issues/188</a>,
  but given the young age of this project, it's possibly a bit out of date.</p>
  
  <p>Not a serious speed test because I'm just dumping and loading a whole
  Dataframe but to give you some impression if you never
  heard of the formats before:</p>
  
  <pre><code> # IPython    
  import numpy as np
  import pandas as pd
  import pyarrow as pa
  import pyarrow.feather as feather
  import pyarrow.parquet as pq
  import fastparquet as fp
  
  
  df = pd.DataFrame({'one': [-1, np.nan, 2.5],
                     'two': ['foo', 'bar', 'baz'],
                     'three': [True, False, True]})
  
  print("pandas df to disk ####################################################")
  print('example_feather:')
  %timeit feather.write_feather(df, 'example_feather')
  # 2.62 ms ± 35.8 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
  print('example_parquet:')
  %timeit pq.write_table(pa.Table.from_pandas(df), 'example.parquet')
  # 3.19 ms ± 51 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
  print()
  
  print("for comparison:")
  print('example_pickle:')
  %timeit df.to_pickle('example_pickle')
  # 2.75 ms ± 18.8 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
  print('example_fp_parquet:')
  %timeit fp.write('example_fp_parquet', df)
  # 7.06 ms ± 205 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)
  print('example_hdf:')
  %timeit df.to_hdf('example_hdf', 'key_to_store', mode='w', table=True)
  # 24.6 ms ± 4.45 ms per loop (mean ± std. dev. of 7 runs, 100 loops each)
  print()
  
  print("pandas df from disk ##################################################")
  print('example_feather:')
  %timeit feather.read_feather('example_feather')
  # 969 µs ± 1.8 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
  print('example_parquet:')
  %timeit pq.read_table('example.parquet').to_pandas()
  # 1.9 ms ± 5.5 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
  
  print("for comparison:")
  print('example_pickle:')
  %timeit pd.read_pickle('example_pickle')
  # 1.07 ms ± 6.21 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
  print('example_fp_parquet:')
  %timeit fp.ParquetFile('example_fp_parquet').to_pandas()
  # 4.53 ms ± 260 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)
  print('example_hdf:')
  %timeit pd.read_hdf('example_hdf')
  # 10 ms ± 43.4 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
  
  # pandas version: 0.22.0
  # fastparquet version: 0.1.3
  # numpy version: 1.13.3
  # pandas version: 0.22.0
  # pyarrow version: 0.8.0
  # sys.version: 3.6.3
  # example Dataframe taken from https://arrow.apache.org/docs/python/parquet.html
  </code></pre>