title: Parallel background tasks on single worker dyno
body: <p>We've got a Rails app where certain requests trigger long-running tasks on a worker dyno with delayed_job, while the front-end polls until it receives a result. </p>
  
  <p>Our traffic is small but growing, and the tasks generally take only a few seconds to complete, but <em>can</em> take up to a minute. Right now a single web and worker dyno each should be sufficient to handle our load. </p>
  
  <p>The problem is, the delayed_job queue won't process jobs in parallel, so a longer task ends up holding up the tasks behind it.</p>
  
  <p>What we're looking for is something like Unicorn for the backend, where a single worker dyno can process multiple tasks concurrently. And since it's Rails, we're looking for something multi-process, not multi-threaded.</p>
  
  <p>(We tried creating multiple worker entries in our procfile- This worked on the local dev box, but not on Heroku)</p>
  
  <p><strong>Procfile:</strong>  </p>
  
  <pre><code>web: bundle exec unicorn -p $PORT -c ./config/unicorn.rb
  # Multiple workers defined here doesn't translate into multiple processes on single worker dyno:
  worker: bundle exec rake jobs:work
  worker: bundle exec rake jobs:work
  worker: bundle exec rake jobs:work
  </code></pre>
  
  <p>This Heroku article proposes using the <a href="https://devcenter.heroku.com/articles/ruby-resque-pool" rel="nofollow">resque-pool gem</a> as a solution. Is that the only solution, or can delayed_job be used for parallel background jobs as well?</p>