title: Decompress, transform and compress back big file in nodejs
body: <p>I have considerably big (30Gb gzipped, ~300Gb gunzipped) gzipped rdf file, which I need to process line by line and gzip back to another file. So here is what I currently have (<a href="https://github.com/dgraph-io/benchmarks/blob/master/data/21million.rdf.gz" rel="nofollow noreferrer">file</a> on test is ~150Mb gzipped)</p>
  
  <pre class="lang-js prettyprint-override"><code>const fs = require('fs');
  const zlib = require('zlib');
  const readline = require('readline');
  
  const readStream = fs.createReadStream('21million.rdf.gz').pipe(zlib.createGunzip());
  const writeStream = fs.createWriteStream("21million.rdf");
  const gzipStream = zlib.createGzip();
  
  gzipStream.pipe(writeStream);
  
  
  const rl = readline.createInterface({
      input: readStream,
      output: gzipStream,
  });
  
  rl.on('line', (line) =&gt; {
      gzipStream.write(`${line.toUpperCase()}\n`);
  });
  
  rl.on('close', () =&gt; {
      console.log('done');
      gzipStream.end();
  });
  </code></pre>
  
  <p>The problem is with such flow I receive <code>FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory</code></p>
  
  <p>So the question is - how can I set it up so I don't hit OOM issue?</p>
  
  <p>PS. I'm aware it can be done with sed, awk, pert etc., yet I need to do it in js.</p>