title: Spring: Creating a truly scalable Thread Pool with ThreadPoolTaskExecutor
body: <p>I have a REST API which uses Spring ThreadPoolTaskExecutor. The API internally submits callable jobs to ThreadPoolTaskExecutor which makes call to third party. Once the call is done, business logic is performed and results are returned back to the caller.</p>
  
  <p>The code works fine but when the load increases then the performance becomes really bad. I am suspecting this to be the result of thread pool size for ThreadPoolTaskExecutor. So say if the concurrent users are n but we have only x number of thread(x is less then n) then the x thread will have to unnecessary wait for limited number of threads to process their request.</p>
  
  <p>I want to process the third party calls in parallel but doesn't want to create a thread pool with huge number of threads.</p>
  
  <p>Options for me is to use  Executors.newFixedThreadPool(y). Use it inside the method and once the process is complete close the object. This is possible but not sure about it side effects like Is it a good practice to create fixed thread pool from the method.</p>
  
  <p>Other option might be to use some kind of object pool like GenericObjectPoolConfig and use it to fetch the threads.</p>
  
  <p>Other option might be to set the max pool size to Integer.max and reduce the queue capacity to 1. So that each time a new request comes instead of storing the object in queue it creates a new thread.</p>
  
  <pre><code>ThreadPoolTaskExecutor threadPoolTaskExecutor = new ThreadPoolTaskExecutor();
      threadPoolTaskExecutor.setCorePoolSize(20);
      threadPoolTaskExecutor.setMaxPoolSize(Integer.MAX_VALUE);
      threadPoolTaskExecutor.setWaitForTasksToCompleteOnShutdown(true);
      threadPoolTaskExecutor.setQueueCapacity(1);
      threadPoolTaskExecutor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
      threadPoolTaskExecutor.initialize();
  </code></pre>
  
  <p>It will be helpful if anyone can share his thoughts.</p>
  
  <pre><code>@Configuration
  public class TestConfiguration{
  
     @Bean
  public ConcurrentTaskExecutor concurrentTaskExecutor() {
      ConcurrentTaskExecutor concurrentTaskExecutor = new ConcurrentTaskExecutor();
      concurrentTaskExecutor.setConcurrentExecutor(getExecutor());
      return concurrentTaskExecutor;
  }
  
  private Executor getExecutor() {
      ThreadPoolTaskExecutor threadPoolTaskExecutor = new ThreadPoolTaskExecutor();
      threadPoolTaskExecutor.setCorePoolSize(20);
      threadPoolTaskExecutor.setMaxPoolSize(30);
      threadPoolTaskExecutor.setWaitForTasksToCompleteOnShutdown(true);
      threadPoolTaskExecutor.setQueueCapacity(75);
      threadPoolTaskExecutor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
      threadPoolTaskExecutor.initialize();
      return threadPoolTaskExecutor;
  }
  }
  
  @Service
  public class TestServiceImpl{
    @Autowired
    private ConcurrentTaskExecutor concurrentTaskExecutor;
  
    @Override
    @Transactional
    public DTO getDTO() {
      Callable&lt;TESTDTO&gt; test1Callable = new Test1Callable();
      Future&lt;TESTDTO&gt; testDTO1 = concurrentTaskExecutor.submit(test1Callable);
  
      Callable&lt;TESTDTO&gt; test2Callable = new Test2Callable();
      Future&lt;TESTDTO&gt; testDTO2 =concurrentTaskExecutor.submit(test2Callable);
  
      Callable&lt;TESTDTO&gt; test3Callable = new Test3Callable();
      Future&lt;TESTDTO&gt; testDTO3 =concurrentTaskExecutor.submit(test3Callable);
  
      // Perform logic on DTO's
      return DTO;
    }
  </code></pre>