title: Spring Batch Runtime Exception in Item processor
body: <p>I am learning spring batch and trying to understand how item processor works, during exception.</p>
  
  <p>I am reading data from csv file in a chunk of 3 records and process it and write it to Database.</p>
  
  <p>my csv file</p>
  
  <pre><code>Jill,Doe
  Joe,Doe
  Justin,Doe
  Jane,Doe
  John,Doem
  Jill,Doe
  Joe,Doe
  Justin,Doe
  Jane,Doe
  </code></pre>
  
  <p>Batch Configuration, reading items in chunk of 3 , and skip limit 2</p>
  
  <pre><code>@Configuration
  @EnableBatchProcessing
  public class BatchConfiguration {
  
      @Autowired
      public JobBuilderFactory jobBuilderFactory;
  
      @Autowired
      public StepBuilderFactory stepBuilderFactory;
  
      @Bean
      public FlatFileItemReader&lt;Person&gt; reader() {
          return new FlatFileItemReaderBuilder&lt;Person&gt;().name("personItemReader").resource(new ClassPathResource("sample-data.csv")).delimited()
                  .names(new String[] { "firstName", "lastName" }).fieldSetMapper(new BeanWrapperFieldSetMapper&lt;Person&gt;() {
                      {
                          setTargetType(Person.class);
                      }
                  }).build();
      }
  
      @Bean
      public PersonItemProcessor processor() {
          return new PersonItemProcessor();
      }
  
      @Bean
      public JdbcBatchItemWriter&lt;Person&gt; writer(DataSource dataSource) {
          return new JdbcBatchItemWriterBuilder&lt;Person&gt;().itemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider&lt;&gt;())
                  .sql("INSERT INTO person (first_name, last_name) VALUES (:firstName, :lastName)").dataSource(dataSource).build();
      }
  
      @Bean
      public Job importUserJob(JobCompletionNotificationListener listener, Step step1) {
          return jobBuilderFactory.get("importUserJob").incrementer(new RunIdIncrementer()).listener(listener).flow(step1).end().build();
      }
  
     @Bean
  public Step step1(JdbcBatchItemWriter&lt;Person&gt; writer) {
      return stepBuilderFactory.get("step1").&lt;Person, Person&gt; chunk(3).reader(reader()).processor(processor()).writer(writer).faultTolerant().skipLimit(2)
              .skip(Exception.class).build();
  }
     }
  </code></pre>
  
  <p>I am trying to simulate a Exception, by throwing Exception manually for one record in my item processor</p>
  
  <pre><code>public class PersonItemProcessor implements ItemProcessor&lt;Person, Person&gt; {
  
      private static final Logger log = LoggerFactory.getLogger(PersonItemProcessor.class);
  
      @Override
      public Person process(final Person person) throws Exception {
          final String firstName = person.getFirstName().toUpperCase();
          final String lastName = person.getLastName().toUpperCase();
  
          final Person transformedPerson = new Person(firstName, lastName);
  
          log.info("Converting (" + person + ") into (" + transformedPerson + ")");
          if (person.getLastName().equals("Doem"))
              throw new Exception("DOOM");
          return transformedPerson;
      }
  }
  </code></pre>
  
  <p>Now as per skip limit, when the exception is thrown, the item processor is re processing the chunk and skips the item which throws error and item write also  inserts all records in DB , except the one record with exception.</p>
  
  <p>This is all fine, because my processor, it is just converting lower to upper case name, and it can be run many times with out impact.</p>
  
  <p>But lets assume if my item processor, is calling web service and sending data.
  and if some exception is thrown after successful calling for web service. then remaining data in the chunk will be processed again (and calling webservice again). 
  I don't want to call web service again, because it is like sending duplicate data to web service and the webservice system cannot identify duplicate data.</p>
  
  <p>How to handle such case. one option is don't skip Exception, which means my still one record in the chunk will not make it to item writer, even though the  processor had called web service. so that is not correct.</p>
  
  <p>other option chunk should be of size 1 , then this may not be efficient in processing thousands of records.</p>
  
  <p>what are the other options ?</p>