title: Spring batch Admin : Unable to read 663 MB flat file, OutOfMemoryError
body: <p>I am using Spring Batch Admin 1.2.1 in Tomcat 7. I am trying to read 663 MB file and getting following error. I have also increased the heap size of Tomcat but of no vain. The job is pretty straight forward. It reads flat file and save it to DB with very little processing. <strong>Please help.</strong></p>
  
  <pre><code>15:05:10,535  INFO http-apr-8181-exec-4 SimpleStepHandler:133 - Executing step: [load]
      15:05:10,981 ERROR http-apr-8181-exec-4 AbstractStep:212 - Encountered an error executing the step
  java.lang.OutOfMemoryError: Java heap space
      at org.apache.catalina.loader.WebappClassLoader.findResourceInternal(WebappClassLoader.java:3098)
      at org.apache.catalina.loader.WebappClassLoader.findResource(WebappClassLoader.java:1244)
      at org.apache.catalina.loader.WebappClassLoader.getResource(WebappClassLoader.java:1407)
      at org.springframework.core.io.ClassPathResource.exists(ClassPathResource.java:139)
      at org.springframework.batch.item.file.FlatFileItemReader.doOpen(FlatFileItemReader.java:248)
      at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:134)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
      at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
      at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
      at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
      at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
      at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
      at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
      at $Proxy30.open(Unknown Source)
      at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:93)
      at org.springframework.batch.core.step.item.ChunkMonitor.open(ChunkMonitor.java:105)
      at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:93)
      at org.springframework.batch.core.step.tasklet.TaskletStep.open(TaskletStep.java:301)
      at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:192)
      at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:135)
      at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:61)
      at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:60)
      at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:144)
      at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:124)
      at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:135)
      at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:281)
      at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:120)
      at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:48)    
  </code></pre>
  
  <p>Following is my setEnv.bat file entry at TOMCAT_HOME\BIN location</p>
  
  <pre><code> set JAVA_OPTS=-DENVIRONMENT=dev -Dlog4j.debug -Xms1024m -Xmx1024m -XX:MaxPermSize=128m  %JAVA_OPTS%
  </code></pre>
  
  <p>Below is my job configs:</p>
  
  <pre><code>&lt;batch:job id="ex_Job" parent="baseJob"&gt;
      &lt;batch:step id="load" parent="baseStep" next="Recon"&gt;
      &lt;batch:tasklet&gt;
              &lt;batch:chunk reader="Reader" writer="Writer" processor="Processor"&gt;
                  &lt;batch:skippable-exception-classes merge="true" /&gt;
              &lt;/batch:chunk&gt;
          &lt;batch:listeners merge="true" /&gt;
          &lt;/batch:tasklet&gt;
     &lt;/batch:step&gt;    
          &lt;batch:listeners merge="true" /&gt;
      &lt;/batch:job&gt;
      &lt;bean id="Reader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step"&gt;
          &lt;property name="linesToSkip" value="1" /&gt;
          &lt;property name="skippedLinesCallback" ref="headerLineCallbackHandler" /&gt;
          &lt;property name="resource" value="${extract.input.data.dir}/#{jobParameters['input.file']}" /&gt;
          &lt;property name="lineMapper"&gt;
              &lt;bean
                  class="org.springframework.batch.item.file.mapping.PatternMatchingCompositeLineMapper"&gt;
                  &lt;property name="tokenizers"&gt;
                      &lt;map&gt;
                          &lt;entry key="*" value-ref="LineTokenizer" /&gt;
                          &lt;entry key="T*" value-ref="TrailerTokenizer" /&gt;
                      &lt;/map&gt;
                  &lt;/property&gt;
                  &lt;property name="fieldSetMappers"&gt;
                      &lt;map&gt;
                          &lt;entry key="*" value-ref="fieldSetMapper" /&gt;
                      &lt;/map&gt;
                  &lt;/property&gt;
              &lt;/bean&gt;
          &lt;/property&gt;
      &lt;/bean&gt;
  
  
      &lt;beans:bean id="fieldSetMapper" class="org.springframework.batch.item.file.mapping.PassThroughFieldSetMapper" /&gt;
      &lt;job id="baseJob" abstract="true"&gt;
          &lt;listeners&gt;
              &lt;listener ref="skipCheckingJobListener"&gt;&lt;/listener&gt;
          &lt;/listeners&gt;
      &lt;/job&gt;
  
      &lt;step id="baseStep" abstract="true"&gt;
          &lt;tasklet&gt;
              &lt;chunk commit-interval="100" skip-limit="10000"&gt;
                  &lt;skippable-exception-classes&gt;
                      &lt;include class="org.springframework.batch.item.file.FlatFileParseException" /&gt;
                      &lt;include class="org.springframework.batch.item.file.transform.IncorrectLineLengthException" /&gt;
                      &lt;include class="org.springframework.dao.DataAccessException"/&gt;
                      &lt;include class="org.springframework.batch.item.file.transform.ConversionException"/&gt;
                  &lt;/skippable-exception-classes&gt;
              &lt;/chunk&gt;
              &lt;listeners&gt;
                  &lt;listener ref="stepExecutionListener" /&gt;
                  &lt;listener ref="genericSkipListener" /&gt;
              &lt;/listeners&gt;
          &lt;/tasklet&gt;
      &lt;/step&gt;
  </code></pre>