title: Best way to read an HDFS file into a string?
body: <p>Currently, I have the following code.</p>
  
  <pre><code>        try
          {           
              FileSystem fs = FileSystem.get(config);
              StringBuilder builder=new StringBuilder();
              byte[] buffer=new byte[4096];
              int bytesRead;
  
              FSDataInputStream in = fs.open(new Path(fname));
              while ((bytesRead = in.read(buffer)) &gt; 0) 
                  builder.append(new String(buffer, 0, bytesRead));
              in.close();
  
              return builder.toString();
          }
          catch (IOException ex) 
          {
              ex.printStackTrace();
              return "";
          }
  </code></pre>
  
  <p>As you can see, I have allocated 4096 bytes to the buffer. Now, since an HDFS block is 64 MB, would it better to allocate a buffer of size 64 MB? What would be the most efficient way to write this, given that my HDFS file size > 64 MB.</p>