title: How to skip Error: field larger than field limit (131072) when using pd.read_csv
body: <p>Loading a 5GB csv file using the following code crashes my pc.</p>
  <pre><code>import pandas as pd
  a = pd.read_csv('file.csv',error_bad_lines=False,encoding='latin1',engine='python')
  </code></pre>
  <p>However, it seems that this code is able to deal with &quot;field larger than field limit&quot; issue as in the log I get messages such as:</p>
  <pre><code>Skipping line 1435768: field larger than field limit (131072)
  </code></pre>
  <p>I tried to overcome my pc limitations by using chunks:</p>
  <pre><code>a=pd.DataFrame()
  for chunk in pd.read_csv('file.csv', error_bad_lines=False,encoding='latin1',engine='python',chunksize=10000):
      a = pd.concat([a, chunk], ignore_index=True)
  </code></pre>
  <p>But this code ignores <code>&quot;error_bad_lines=False&quot;</code> as it stops with the following error message:</p>
  <pre><code>Error: field larger than field limit (131072)
  </code></pre>
  <p>I also tried to read only part of the file, but again error lines are not being skipped:</p>
  <pre><code>a = pd.read_csv('file.csv', error_bad_lines=False,encoding='latin1',engine='python',skiprows=0, nrows=1000000)
  </code></pre>
  <p>results in :</p>
  <pre><code>Error: field larger than field limit (131072)
  </code></pre>
  <p>Any suggestions how to overcome this problem? I'd like to skip these problematic lines.</p>