title: Batch with Spring Boot & JPA - use in-memory datasource for batch-related tables
body: <p><strong>Context</strong></p>
  
  <p>I'm trying to develop a batch service with Spring Boot, using JPA Repository. Using two different datasources, I want the batch-related tables created in a in-memory database, so that it does not pollute my business database.</p>
  
  <p>Following multiple topics on the web, I came up with this configuration of my two datasources :</p>
  
  <pre><code>@Configuration
  public class DataSourceConfiguration {
  
      @Bean(name = "mainDataSource")
      @Primary
      @ConfigurationProperties(prefix="spring.datasource")
      public DataSource mainDataSource(){
          return DataSourceBuilder.create().build();
      }
  
      @Bean(name = "batchDataSource")
      public DataSource batchDataSource( @Value("${batch.datasource.url}") String url ){
          return DataSourceBuilder.create().url( url ).build();
      }   
  }
  </code></pre>
  
  <p>The first one, <code>mainDataSource</code>, uses the default Spring database configuration. The <code>batchDataSource</code> defines an embedded HSQL database, in which I want the batch and step tables to be created.</p>
  
  <pre><code># DATASOURCE (DataSourceAutoConfiguration &amp; DataSourceProperties)
  spring.datasource.url=jdbc:mariadb://localhost:3306/batch_poc
  spring.datasource.username=root
  spring.datasource.password=
  spring.datasource.driver-class-name=org.mariadb.jdbc.Driver
  spring.datasource.max-age=10000
  
  spring.datasource.initialize=false
  
  # JPA (JpaBaseConfiguration, HibernateJpaAutoConfiguration)
  spring.jpa.generate-ddl=false
  spring.jpa.show-sql=true
  spring.jpa.database=MYSQL
  
  # SPRING BATCH (BatchDatabaseInitializer)
  spring.batch.initializer.enabled=false
  
  # ----------------------------------------
  # PROJECT SPECIFIC PROPERTIES
  # ----------------------------------------
  
  # BATCH DATASOURCE
  batch.datasource.url=jdbc:hsqldb:file:C:/tmp/hsqldb/batchdb
  </code></pre>
  
  <p>Here is my batch config :</p>
  
  <pre><code>@Configuration
  @EnableBatchProcessing
  public class BatchConfiguration {
  
      private static final Logger LOG = Logger.getLogger( BatchConfiguration.class );
  
      @Bean
      public BatchConfigurer configurer(){
          return new CustomBatchConfigurer();
      }
  
      @Bean
      public Job importElementsJob( JobBuilderFactory jobs, Step step1 ){
          return jobs.get("importElementsJob")
                  .incrementer( new RunIdIncrementer() )
                  .flow( step1 )
                  .end()
                  .build();               
      }
  
      @Bean
      public Step step1( StepBuilderFactory stepBuilderFactory, ItemReader&lt;InputElement&gt; reader,
              ItemWriter&lt;List&lt;Entity&gt;&gt; writer, ItemProcessor&lt;InputElement, List&lt;Entity&gt;&gt; processor ){
  
          return stepBuilderFactory.get("step1")
                  .&lt;InputElement, List&lt;Entity&gt;&gt; chunk(100)
                  .reader( reader )
                  .processor( processor )
                  .writer( writer )
                  .build();
      }
  
      @Bean
      public ItemReader&lt;InputElement&gt; reader() throws IOException {       
          return new CustomItemReader();
      }
  
      @Bean
      public ItemProcessor&lt;InputElement, List&lt;Entity&gt;&gt; processor(){
          return new CutsomItemProcessor();
      }
  
      @Bean
      public ItemWriter&lt;List&lt;Entity&gt;&gt; writer(){
          return new CustomItemWriter();
      }
  
  }
  </code></pre>
  
  <p>The BatchConfigurer, using the in-memory database :</p>
  
  <pre><code>public class CustomBatchConfigurer extends DefaultBatchConfigurer {
  
      @Override
      @Autowired
      public void setDataSource( @Qualifier("batchDataSource") DataSource dataSource) {
          super.setDataSource(dataSource);
      }
  
  }
  </code></pre>
  
  <p>And, finally, my writer :</p>
  
  <pre><code>public class CustomItemWriter implements ItemWriter&lt;List&lt;Entity&gt;&gt; {
  
      private static final Logger LOG = Logger.getLogger( EntityWriter.class );
  
      @Autowired
      private EntityRepository entityRepository;
  
      @Override
      public void write(List&lt;? extends List&lt;Entity&gt;&gt; items)
              throws Exception {
          if( items != null &amp;&amp; !items.isEmpty() ){
  
              for( List&lt;Entity&gt; entities : items ){
                  for( Entity entity : entities ){                        
                      Entity fromDb = entityRepository.findById( entity.getId() );
  
                      // Insert
                      if( fromDb == null ){
                          entityRepository.save( entity );
                      }
  
                      // Update
                      else {
                          // TODO : entityManager.merge()
                      }
                  }
              }
  
          }
      }
  
  }
  </code></pre>
  
  <p>The EntityRepository interface extends JpaRepository.</p>
  
  <p><strong>Problem</strong></p>
  
  <p>When I separate the datasources this way, nothing happens when I call the save method of the repository. I see the select queries from the call of findById() in the logs. But nothing for the save. And my output database is empty at the end. </p>
  
  <p>When I come back to a unique datasource configuration (removing the configurer bean and letting Spring Boot manage the datasource alone), the insert queries work fine.</p>
  
  <p>Maybe the main datasource configuration is not good enough for JPA to perform the inserts correctly. But what is missing ?</p>