title: Read spark csv dataframe as pandas
body: <p>After processing a big data on pyspark, I saved it on csv using the following command:</p>
  <pre><code>df.repartition(1).write.option(&quot;header&quot;, &quot;true&quot;).option(&quot;delimeter&quot;, &quot;\t&quot;).csv(&quot;csv_data&quot;, mode=&quot;overwrite&quot;)
  </code></pre>
  <p>Now, I want use <code>pd.read_csv()</code> to load it again.</p>
  <pre><code>info = pd.read_csv('part0000.csv', sep='\t', header='infer')
  </code></pre>
  <p><code>info</code> is returned as 1 column where the data is separated by comma not '\t'.</p>
  <pre><code>col1name,col2name,col3name
  val1,val2,val3
  </code></pre>
  <p>I tried to specify the <code>sep=','</code> but I got an parsing error where some rows have more than 3 cols.</p>
  <p>How to fix that without skipping any rows ? Is there anything to do with spark to resolve it such as specify a <code>'|'</code> as delimiter</p>