title: Gzip file reader Scala
body: <p>I'm new to scala and figuring out things on the fly. I have a program that needs to read Gzip files of various sizes - 20KB, 2MB and 150MB(Yes, the zipped file is 150MB). I would think not to have a different approach for reading different files, but a standard one through-out. Most of the approaches that I see use a buffer size of 64MB to read files line by line? What is the best( read as, *<em>fastest and clean memory *</em> way of doing it) way to do this ? </p>
  
  <p>Thanks in advance,for the help!</p>
  
  <p><strong>update 1:</strong> </p>
  
  <p>Great improvments in reading rate.(I would even share my karma points) Thanks SO ! :) </p>
  
  <p>But, I noticed that, since each of my file has around 10K lines, while writing them to file, It takes a long time to convert the String Iterator to a string before writing to file.I can do two approaches, </p>
  
  <ol>
  <li>Iterator line by line and write line by line to file. </li>
  <li>Iterate line by line to convert the lines to a big string ("\n" delimited) and write that big string to file.</li>
  </ol>
  
  <p>I'm assuming [2] would be faster. So, this is what am doing for writing, </p>
  
  <pre><code>var processedLines = linesFromGzip(new File(fileName)).map(line =&gt; MyFunction(line))
  
  var  outFile = Resource.fromFile(outFileName)
  
  outFile.write(processedLines.mkString("\n"))  // severe overhead -&gt; processedLines.mkString("\n")
  </code></pre>
  
  <p>Also my analysis( by commenting the write() shows that, it doesn't take much of time to write but to convert the <code>processedLines</code> to a Single big String - It takes close to a second - which is huge cost for my application. What would the best(again clean without any memory leaks) way to do this.  </p>