title: What is the optimal way to call N blocking api calls in parallel to get most out of CPU while waiting for api calls to complete?
body: <p>I am working on a micro-service which have following flow of operations :</p>
  
  <p><a href="https://i.stack.imgur.com/iph5W.png" rel="noreferrer"><img src="https://i.stack.imgur.com/iph5W.png" alt="enter image description here"></a></p>
  
  <p>A request comes to execute some number of tasks. After some pre-processing I know that I need to execute some number of tasks let's say 10. Now the tasks are independent of each other, so can be executed in parallel. And each task have some processing step and some external API calls.  And after all tasks are completed the combined results need to be returned. </p>
  
  <p>This is one request, so obviously this micro service can get many such requests in parallel too. </p>
  
  <p>Here API calls are most time-consuming operations and other work executes in very less time comparatively. So I want to design this in a way that as many tasks can be executed in parallel because for tasks mostly would be blocked for API calls.</p>
  
  <p>A simple solution I see is using a thread-pool using ExecutorService, but it doesn't seem like a ideal solution because let's say I create a thread pool of 32 threads, and I get 60 tasks. So only 32 would be executing at a time, even though those 32 tasks are blocked for api call and not using much CPU time.</p>
  
  <p>Is this possible to achieve without breaking the task as a single unit ? </p>