title: Getting the error : "SystemError: Negative size passed to PyString_FromStringAndSize"
body: <p>I am trying to parse huge JSON file (around 20GB). 
  Trying to read one line at a time (each line is a JSON object) and extract the required details.</p>
  
  <p>Example:</p>
  
  <p>JSON file data looks like the one shown below </p>
  
  <pre><code>{
      {a: [], b: [], c: [], d: [],e: []},
      {a: [], b: [], c: [], d: [],e: []},
      .....,
      {a: [], b: [], c: [], d: [],e: []}, 
  }
  </code></pre>
  
  <p>Snippet to parse:</p>
  
  <pre><code>count = 0;
  with open(fileName) as fp:
      try:
          for line in fp:
          data_local = json.loads(line)
          count = count + 1
          #access the data_local["a"]
      except:
          print "Error found" , count , len(data_local["a"])
  </code></pre>
  
  <p>Error Message (when "except block" not used):</p>
  
  <pre><code>Traceback (most recent call last):
  File "./xyzFile", line 606, in &lt;module&gt;
  for line in fp:
  SystemError: Negative size passed to PyString_FromStringAndSize
  </code></pre>
  
  <p>Output (when "except" block" is used)</p>
  
  <pre><code>Error found  65 5392287
  </code></pre>
  
  <p>Found  <a href="https://stackoverflow.com/questions/16326793/is-there-a-maximum-size-for-pythons-shelve-module/16327130#16327130">something similar </a> on stack overflow but that didn't help. Tried to debug by catching the exception. It throws the error after reading 65th JSON objects (lines). Each JSON object is huge(in size and no of values)</p>
  
  <p>Any lead on this would be appreciated.</p>
  
  <p>Thanks</p>