title: Docker/Kubernetes + Gunicorn/Celery - Multiple Workers vs Replicas?
body: <p>I was wondering what the correct approach to deploying a containerized Django app using gunicorn &amp; celery was. </p>
  
  <p>Specifically, each of these processes has a built-in way of scaling vertically, using <code>workers</code> for gunicorn and <code>concurrency</code> for celery. And then there is the Kubernetes approach to scaling using <code>replicas</code></p>
  
  <p>There is also this notion of setting workers equal to some function of the CPUs. Gunicorn recommends </p>
  
  <blockquote>
    <p>2-4 workers per core</p>
  </blockquote>
  
  <p>However, I am confused what this translates to on K8s where CPU is a divisible shared resource - unless I use resoureceQuotas.</p>
  
  <p>I want to understand what the Best Practice is. There are three options I can think of:</p>
  
  <ul>
  <li>Have single workers for gunicorn and a concurrency of 1 for celery, and scale them using the replicas? (horizontal scaling)</li>
  <li>Have gunicorn &amp; celery run in a single replica deployment with internal scaling (vertical scaling). This would mean setting fairly high values of workers &amp; concurrency respectively. </li>
  <li>A mixed approach between 1 and 2, where we run gunicorn and celery with a small value for workers &amp; concurrency, (say 2), and then use K8s Deployment replicas to scale horizontally.</li>
  </ul>
  
  <p>There are some questions on SO around this, but none offer an in-depth/thoughtful answer. Would appreciate if someone can share their experience.</p>
  
  <p>Note: We use the default worker_class <code>sync</code> for Gunicorn</p>