title: Java Web Server - Parallel workers design VS assemply line design for step by step IO operations
body: <p>I'm writing a java server interface which contains several steps of http requests one by one, each of which need about 10 ms to finish, and the final result won't be returned to the client. The method is like:</p>
  
  <pre><code>public void printUserBookList(int userId){
    var user = userService.getUser(userId); // http req taking 10 ms to finish
    var bookShelf = bookShelfService.getShelf(user); // http req taking 10 ms to finish
    var books = bookService.getBooks(bookShelf); // http req taking 10 ms to finish
    printService.print(books); // http req taking 10 ms to finish
  }
  </code></pre>
  
  <p>For better throughput and performance, which concurrency model should I use?</p>
  
  <ol>
  <li><strong>parallel workers design</strong> (All steps executed synchronously one by one)</li>
  </ol>
  
  <p>Whenever there is an incoming request, the tomcat server would automatically assign the task to a thread(if possible). The task itself is executed synchronously. For example, if there is totally 4 steps, each taking 10 ms to finish, then it will take 40 ms to complete this task, and then the worker thread will be freed to accept another task. This design requires longer time for every worker to finish its task.</p>
  
  <ol start="2">
  <li><strong>assembly line design</strong> (All steps executed asynchronously one by one)</li>
  </ol>
  
  <p>The steps in a task would be handled by different actors. Each actor has its own backing thread pool. All steps inside a task are executed asynchronously, so every top level worker can complete its task momentarily after receiving one. But this design requires more <code>ThreadPoolExecutors</code> at each step</p>
  
  <p>The actor-style design:</p>
  
  <pre><code>public void printUserBookList(int userId){
    userActor.getUserAndThenGoNextStepAsync(userId);
    // inside getUserAndThenGoNextStepAsync, IO operations are executed by another threadPoolExecutor
    // and the result will be passed to the successors for further async executions
  }
  </code></pre>
  
  <p>For an ordinary tomcat server, which design could promise better throughput and performance? And would the all-async design better exploit the CPU usage?</p>