title: How to read big json?
body: <p>I receive json-files with data to be analyzed in R, for which I use the RJSONIO-package:</p>
  
  <pre><code>library(RJSONIO)
  filename &lt;- "Indata.json"
  jFile &lt;- fromJSON(filename)
  </code></pre>
  
  <p>When the json-files are larger than about 300MB (uncompressed), my computer starts to use the swap memory and continues the parsing (fromJSON) for hours. A 200MB-file takes only about one minute to parse.</p>
  
  <p>I use R 2.14 (64bit) on Ubuntu 64bit with 16GB RAM, so I'm surprised that swapping is needed already at about 300MB of json.</p>
  
  <p>What can I do to read big jsons? Is there something in the memory-settings that mess things up? I have restarted R and run only the three lines above. The json-file contain 2-3 columns with short strings, and 10-20 columns with numbers from 0 to 1000000. I.e. it is the number of rows that makes the large size (more than a million rows in the parsed data).</p>
  
  <hr>
  
  <p><strong>Update:</strong> From the comments I learned that rjson is done more in C, so I tried it. A 300MB file that with RJSONIO (according to Ubuntu System Monitor) reached 100% memory use (from 6% baseline) and went on to swapping, needed only 60% memory with package rjson and the parsing was done in reasonable time (minutes).</p>