title: Spring Batch 'RunIdIncrementer' not generating next value
body: <p>I have several Spring Batch (2.1.9.RELEASE) jobs running in production that use <code>org.springframework.batch.core.launch.support.RunIdIncrementer</code>.</p>
  
  <p>Sporadically, I get the following error:</p>
  
  <pre><code>org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException: A job instance already exists and is complete for parameters={run.id=23, tenant.code=XXX}.  If you want to run this job again, change the parameters.
      at org.springframework.batch.core.repository.support.SimpleJobRepository.createJobExecution(SimpleJobRepository.java:122) ~[spring-batch-core-2.1.9.RELEASE.jar:na]
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.6.0_39]
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) ~[na:1.6.0_39]
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_39]
      at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_39]
      at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318) ~[spring-aop-3.1.1.RELEASE.jar:3.1.1.RELEASE]
      at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) ~[spring-aop-3.1.1.RELEASE.jar:3.1.1.RELEASE]
      at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) ~[spring-aop-3.1.1.RELEASE.jar:3.1.1.RELEASE]
      at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:110) ~[spring-tx-3.1.1.RELEASE.jar:3.1.1.RELEASE]
      at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) ~[spring-aop-3.1.1.RELEASE.jar:3.1.1.RELEASE]
      at org.springframework.batch.core.repository.support.AbstractJobRepositoryFactoryBean$1.invoke(AbstractJobRepositoryFactoryBean.java:168) ~[spring-batch-core-2.1.9.RELEASE.jar:na]
      at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) ~[spring-aop-3.1.1.RELEASE.jar:3.1.1.RELEASE]
      at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202) ~[spring-aop-3.1.1.RELEASE.jar:3.1.1.RELEASE]
      at sun.proxy.$Proxy64.createJobExecution(Unknown Source) ~[na:na]
      at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:111) ~[spring-batch-core-2.1.9.RELEASE.jar:na]
      at org.springframework.batch.core.launch.support.CommandLineJobRunner.start(CommandLineJobRunner.java:349) [spring-batch-core-2.1.9.RELEASE.jar:na]
      at org.springframework.batch.core.launch.support.CommandLineJobRunner.main(CommandLineJobRunner.java:574) [spring-batch-core-2.1.9.RELEASE.jar:na]
      at (omitted for brevity)
  </code></pre>
  
  <p>A sampling from the various XML contexts:</p>
  
  <pre><code>&lt;bean
      id="jobParametersIncrementer"
      class="org.springframework.batch.core.launch.support.RunIdIncrementer" /&gt;
  
  &lt;batch:job id="rootJob"
      abstract="true"
      restartable="true"&gt;
      &lt;batch:validator&gt;
          &lt;bean class="org.springframework.batch.core.job.DefaultJobParametersValidator"&gt;
              &lt;property name="requiredKeys" value="tenant.code"/&gt;
          &lt;/bean&gt;
      &lt;/batch:validator&gt;
  &lt;/batch:job&gt;
  
  &lt;batch:job id="rootJobWithIncrementer"
      abstract="true"
      parent="rootJob"
      incrementer="jobParametersIncrementer" /&gt;
  </code></pre>
  
  <p>I use <code>org.springframework.batch.core.launch.support.CommandLineJobRunner</code> to execute the job:</p>
  
  <pre><code>java org.springframework.batch.core.launch.support.CommandLineJobRunner /com/XXX/job123/job123-context.xml job123 tenant.code=XXX -next 
  </code></pre>
  
  <p>All of the jobs (that use the incrementer) have <code>rootJobWithIncrementer</code> as parent.</p>
  
  <p>I did quite a bit of research and found that some who got this error had success changing the isolation level of the transaction manager. I fiddled with several levels, finally arriving at <code>READ_COMMITED</code>.</p>
  
  <pre><code>&lt;batch:job-repository
      id="jobRepository"
      data-source="oracle_hmp"
      transaction-manager="dataSourceTransactionManager"
      isolation-level-for-create="READ_COMMITTED"/&gt;
  </code></pre>
  
  <p>Based on my understanding, this type of error should only happen if the <em>same</em> job is executed at the <em>same</em> time from multiple processes -- so that there might be contention for the incrementer. In this instance, that is <strong>not</strong> the case, yet we see the error.</p>
  
  <p>Any ideas as to what might be causing this problem? Should I try a different isolation level? Something else?</p>
  
  <p>Thanks!</p>
  
  <p><em>There is a similar question <a href="https://stackoverflow.com/questions/12976662/springbatch-commandlinejobrunner-next-using-old-run-id">here</a>, but it is not as well documented (and also lacks and answer).</em></p>