title: Read bulk data from file (XML/Flat/CSV) and upload data in database
body: <p>I need to read around 1 000 000 lines of data from a file and upload the data in the database using Java.
  Below are the details of requirement and some of the solution I've already written:</p>
  
  <ol>
  <li><p>Read data from a file (any format XML, flat file, CSV).
  Depending upon the file format initialize the class (using Factory pattern). For text files I am using BufferedReader and then add each object created e.g. employee in a list.
  Similarly for XML I am using SAXParser and same as above after reading data from one object it is added to a list.</p></li>
  <li><p>But I don't think we can save or we should be saving 1 000 000 objects in a list. So I need to flush data into data base parallely as the file is being read and list reaches a certain size. This is where I am stuck. How should this be handled? Should I run a parallel Thread which would update data in database once list size reaches say 1000 and meanwhile the thread reading the file should be put on wait?
  DB can be any database Oracle/ MS access.</p></li>
  </ol>
  
  <p>Below is the code which is reading text file:</p>
  
  <pre><code>public class FlatFileLoader implements Loader {
  
  @Override
  public List&lt;Employee&gt; readData(String path) {
  
      try {
          List&lt;Employee&gt; personList =  new ArrayList&lt;Person&gt;();
          FileReader fr = new FileReader(path);
          BufferedReader br = new BufferedReader(fr);
            String line=null;
  
            while ((line = br.readLine()) != null) {
                Employee person = scanLine(line, DataConstants.DELIMITER);
                if(null != person)
                personList.add(person);
  
              }
            br.close();
  
          }
          catch (IOException e) {
            System.err.println(e);
          }
      return null;
  }
  
  private Employee scanLine(String recordLine, String delim) {
      Employee person=null;
      List data= new ArrayList();
      StringTokenizer lineScanner = new StringTokenizer(recordLine, delim);
      while (lineScanner.hasMoreElements()) {
          String nextEle= (String) lineScanner.nextElement();
          if(nextEle.equalsIgnoreCase("FIRST_NAME")){
              break;
          }
          data.add(nextEle);
  
      }
      if(data.size() &gt; 0){
          DataUtil.populatedata(data);
      }
      return person;
  }
  
  }
  </code></pre>
  
  <p>Method
  <code>DataUtil.populatedata(data)</code>
  creates <code>Employee</code> object.
  So where should I put the call for updating data in DB?
  Any suggestion for improving above code is also welcome.</p>