title: Why is pandas.read_sas() so much slower than pandas.read_csv()?
body: <p>I'm writing a Python script to do some data work for me and I have a question about performance. I'm running Python 3.6 and the most up-to-date package of pandas. OS is Windows 10. Anyway here's the jist - I'm reading in a SAS table of about 71 million rows and 39 columns in this way:</p>
  
  <pre><code>want = []
  for chuck in pd.read_sas('my_sas_table.sas7bdat',chunksize = 50000,encoding='cp1252'):
      chuck['DT_COL'] = pd.to_timedelta(chuck['DT_COL'], unit='D') + pd.Timestamp('1960-1-1')
      want.append(chuck)
  </code></pre>
  
  <p>This is taking about 35-40 minutes to read in the table. If I were to go into SAS, export the table as a CSV and than just do </p>
  
  <pre><code>pd.read_csv('my_sas_table.csv') 
  </code></pre>
  
  <p>it only takes about 5 minutes. I don't want to do this because this script will be automated - and I'm trying to avoid doing anything outside of Python. Is there a trick I might be missing or can someone explain this crazy difference in performance?</p>
  
  <p>I know changing the date column on the chunks might be slowing down the read a little bit - but I'm just not convinced that it's causing the </p>
  
  <pre><code>pd.read_sas() 
  </code></pre>
  
  <p>to be 8x slower than </p>
  
  <pre><code>pd.read_csv()
  </code></pre>
  
  <p>but I'm no expert and I'm hoping one of you might be!</p>
  
  <p>Thanks!</p>