title: How to avoid OutOfMemory exception while reading large files in Java
body: <p>I am working on the application which reads large amounts of data from a file. Basically, I have a huge file (around 1.5 - 2 gigs) containing different objects (~5 to 10 millions of them per file). I need to read all of them and put them to different maps in the app. The problem is that the app runs out of memory while reading the objects at some point. Only when I set it to use -Xmx4096m - it can handle the file. But if the file will be larger, it won't be able to do that anymore. </p>
  
  <p>Here's the code snippet:</p>
  
  <pre><code>String sampleFileName = "sample.file";
  FileInputStream fileInputStream = null;
  ObjectInputStream objectInputStream = null;
  try{
      fileInputStream = new FileInputStream(new File(sampleFileName));
      int bufferSize = 16 * 1024;
      objectInputStream = new ObjectInputStream(new BufferedInputStream(fileInputStream, bufferSize));
          while (true){
              try{
                  Object objectToRead = objectInputStream.readUnshared();
                  if (objectToRead == null){
                      break;
                  }
                  // doing something with the object
              }catch (EOFException eofe){
                  eofe.printStackTrace();
                  break;
              } catch (Exception e) {
                  e.printStackTrace();
                  continue;
              }
          }
  } catch (Exception e){
          e.printStackTrace();
  }finally{
      if (objectInputStream != null){
          try{
              objectInputStream.close();
          }catch (Exception e2){
              e2.printStackTrace();
          }
      }
      if (fileInputStream != null){
          try{
              fileInputStream.close();
          }catch (Exception e2){
              e2.printStackTrace();
          }
      }
  }
  </code></pre>
  
  <p>First of all, I was using <em>objectInputStream.readObject()</em> instead of <em>objectInputStream.readUnshared()</em>, so it solved the issue partially. When I increased the memory from 2048 to 4096, it started parsing the file. BufferedInputStream is already in use. From the web I've found only examples how to read lines or bytes, but nothing regarding objects, performance wise.</p>
  
  <p>How can I read the file without increasing the memory for JVM and avoiding the OutOfMemory exception? Is there any way to read objects from the file, not keeping anything else in the memory? </p>