title: Unresponsive scala-play application
body: <p>I am using scala-play app in production. Few days back I observed that because of high CPU at DB side, play app started acting up and response time increased up-to few minutes. Play app was deployed on 3 EC2 instances and all of them were attached to ELB. During this time two processes went unresponsive and response time went up-to 600 minutes(usually response-time is below 200 mili-seconds). Because of high response-time at two of the processes, ELB marked them as unhealthy and all requests were routed to single process(which had response time of 20 seconds). Going through logs didn't find help much. After exploring few articles, I understood that deadlock in thread-pool can be one of the reason. We have used thread-pool for blocking S3 calls and non-blocking DB calls. Different thread-pool is used for these purposes.</p>
  
  <pre>
  
      executor {
        sync = {
          fork-join-executor {
            parallelism-factor = 1.0
            parallelism-max = 24
          }
        }
  
        async = {
          fork-join-executor {
            parallelism-factor = 1.0
            parallelism-max = 24
          }
        }
      }
  
  </pre>
  
  <p>Can anyone help in understanding what could possibly have gone wrong?
  All 3 nodes have same build deployed, but only two of them went unresponsive. CPU at these unresponsive nodes was less than 10%.</p>
  
  <p>Play: 2.5.14
  Scala: 2.11.11</p>