title: Write large file in Python with problems
body: <p>I am dealing with ConceptNet Json files, and trying to extract relationships between entities in the files. The file is very large, for example, <code>part_00.json</code> is <code>1.4Gb</code>, I write a Python program to handle the large file and try to save the processed data into a new output file. The source code is provided below:</p>
  
  <pre><code>import json
  
  inputData = "d:\\part_00.jsons"
  outputData = "d:\\output.jsons"
  fin = open(inputData, 'r')  
  fout = open(outputData, 'w')
  
  outputfile =[]
  for eachLine in fin:  
      line = eachLine.strip()
      line = line.strip(',')                                   
      js = None  
      try:  
          js = json.loads(line)  
          outputfile.append(js["uri"])   
      except Exception,e:  
          print 'bad line'  
          continue
  
  for line in outputfile:
      if line.startswith("/a"):
          a = line[4:-1]
          r, c1, c2 = a.split(",")
          if c1.split("/")[2] == "en" and c2.split("/")[2] == "en":
              fout.write((" ".join((r, c1.split("/")[3], c2.split("/")[3]))).encode('utf8'))
              fout.write("\n")    
  
  fout.close()  
  </code></pre>
  
  <p>However, when I run the program, I got error:</p>
  
  <pre><code>Traceback (most recent call last):
    File "D:\workspace\CSK\src\readJson.py", line 48, in &lt;module&gt;
      r, c1, c2 = a.split(",")
  ValueError: too many values to unpack
  </code></pre>
  
  <p>Then I checked the output file <code>output.jsons</code> and found that it has only <code>57mb</code> size and incomplete. </p>
  
  <p>I do not know what is the cause of the problem. Is it due to the large size of the file? Or there are some cache error? I wish to have your answers.</p>