title: Spring Batch - How to generate parallel steps based on params created in a previous step
body: <h2>Introduction</h2>
  
  <p>I am trying to use jobparameters created in a tasklet to create steps following the execution of the tasklet.</p>
  
  <p>A tasklet tries to finds some files (findFiles()) and if it finds some files it saves the filenames to a list of strings.</p>
  
  <p>In the tasklet I pass the data as following:
      <code>chunkContext.getStepContext().getStepExecution().getExecutionContext().put("files", fileNames);</code></p>
  
  <p>The next step is a parallel flow where for each file a simple reader-processor-writer step will be executed (if you are interested in how I got there please see my previous question: <a href="https://stackoverflow.com/questions/37238813/spring-batch-looping-a-reader-processor-writer-step">Spring Batch - Looping a reader/processor/writer step</a>)</p>
  
  <p>Upon building the job readFilesJob() a flow is created initially using a "fake" list of files because only after the tasklet has been executed the real list of files is known.</p>
  
  <h2>Question</h2>
  
  <p><em>How do I configure my job so the tasklet gets executed first and then the parallel flow gets executed using the list of files generated from the tasklet?</em></p>
  
  <p>I think it comes down to getting the list of filenames loaded with the correct data at the correct moment during runtime... but how?</p>
  
  <h2>Reproduce</h2>
  
  <p>Here is my simplified configuration:</p>
  
  <pre><code>@Configuration
  @EnableBatchProcessing
  public class BatchConfiguration {
  
      private static final String FLOW_NAME = "flow1";
      private static final String PLACE_HOLDER = "empty";
  
      @Autowired
      public JobBuilderFactory jobBuilderFactory;
  
      @Autowired
      public StepBuilderFactory stepBuilderFactory;
  
      public List&lt;String&gt; files = Arrays.asList(PLACE_HOLDER);
  
      @Bean
      public Job readFilesJob() throws Exception {   
          List&lt;Step&gt; steps = files.stream().map(file -&gt; createStep(file)).collect(Collectors.toList());
  
          FlowBuilder&lt;Flow&gt; flowBuilder = new FlowBuilder&lt;&gt;(FLOW_NAME);
  
          Flow flow = flowBuilder
                  .start(findFiles())             
                  .next(createParallelFlow(steps))
                  .build();       
  
          return jobBuilderFactory.get("readFilesJob")                
                  .start(flow)                
                  .end()
                  .build();
      }
  
      private static Flow createParallelFlow(List&lt;Step&gt; steps){
          SimpleAsyncTaskExecutor taskExecutor = new SimpleAsyncTaskExecutor();
          taskExecutor.setConcurrencyLimit(steps.size());
  
          List&lt;Flow&gt; flows = steps.stream()
                  .map(step -&gt;
                          new FlowBuilder&lt;Flow&gt;("flow_" + step.getName()) 
                          .start(step) 
                          .build()) 
                  .collect(Collectors.toList());
  
          return new FlowBuilder&lt;SimpleFlow&gt;("parallelStepsFlow").split(taskExecutor) 
               .add(flows.toArray(new Flow[flows.size()]))
               .build();      
      }
  
      private Step createStep(String fileName){
          return stepBuilderFactory.get("readFile" + fileName)
                  .chunk(100)
                  .reader(reader(fileName))               
                  .writer(writer(filename))                               
                  .build();
      }
  
      private FileFinder findFiles(){
          return new FileFinder();
      }
  }
  </code></pre>
  
  <h2>Research</h2>
  
  <p>The question and answer from <a href="https://stackoverflow.com/questions/29776901/how-to-safely-pass-params-from-tasklet-to-step-when-running-parallel-jobs">How to safely pass params from Tasklet to step when running parallel jobs</a> suggest the usage of a construct like this in the reader/writer: </p>
  
  <p><code>@Value("#{jobExecutionContext[filePath]}") String filePath</code></p>
  
  <p>However, I really hope it is possible to pass the fileName as a string to the reader/writer due to the way the steps are created in the createParallelFlow() method. Therefore, even tho the answer to that question might be a solution for my problem here, it is not the desired solution. But please do not refrain from correcting me if I am wrong.</p>
  
  <h2>Closing</h2>
  
  <p>I am using the file names example to clarify the problem better. My problem is not actually the reading of multiple files from a directory. My question really boils down to the idea of generating data during runtime and passing it to the next dynamically generated step(s).</p>
  
  <h2>EDIT:</h2>
  
  <p>Added a simplified tasklet of the fileFinder.</p>
  
  <pre><code>@Component
  public class FileFinder implements Tasklet, InitializingBean {
  
      List&lt;String&gt; fileNames;
  
      public List&lt;String&gt; getFileNames() {
          return fileNames;
      }
  
      @PostConstruct
      public void afterPropertiesSet() {
          // read the filenames and store dem in the list
          fileNames.add("sample-data1.csv");
          fileNames.add("sample-data2.csv");
      }
  
      @Override
      public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
          // Execution of methods that will find the file names and put them in the list...
          chunkContext.getStepContext().getStepExecution().getExecutionContext().put("files", fileNames);                     
          return RepeatStatus.FINISHED;
      }    
  }
  </code></pre>