title: Thread Management Best Practices for Network Bound Web Server
body: <p>I have a network server that has a few dozen backend controllers, each which process a different user request (i.e., user clicking something on the website).</p>
  <p>Each controller makes network calls to a handful of services to get the data it needs. Each network call takes somewhere around 200ms. These network calls can be done in parallel, so I want to launch one thread for each of them and then collect at the end - maximizing parallelization. (5 network calls in parallel takes 200ms, where 5 in sequence will take 1000ms).</p>
  <p>However I am unsure of best practice to design the thread management strategy.</p>
  <ol>
  <li>Should I have one threadpool with say 1000 (arbitrary number for example) threads in it, and each controller draws from that pool?</li>
  <li>Should I not have a threadpool at all and create new threads in each controller as I need them? This option seems &quot;dumb&quot; but I wonder - how much is the cost in CPU cycles of creating a thread, compared to waiting for network response? Quite minimal.</li>
  <li>Should I have one threadpool per controller? (Meaning dozens of threadpools, each with around 5 or 6 threads for that specific controller).</li>
  </ol>
  <p>Seeking pros / cons of each strategy, best practices, or an alternate strategy I haven't considered.</p>