title: How to restart Celery gracefully without delaying tasks
body: <p>We use Celery with our Django webapp to manage offline tasks; some of these tasks can run up to 120 seconds.</p>
  
  <p>Whenever we make any code modifications, we need to restart Celery to have it reload the new Python code.  Our current solution is to send a SIGTERM to the main Celery process (<code>kill -s 15 `cat /var/run/celeryd.pid`</code>), then to wait for it to die and restart it (<code>python manage.py celeryd --pidfile=/var/run/celeryd.pid [...]</code>).</p>
  
  <p>Because of the long-running tasks, this usually means the shutdown will take a minute or two, during which no new tasks are processed, causing a noticeable delay to users currently on the site.  I'm looking for a way to tell Celery to shutdown, but then immediately launch a new Celery instance to start running new tasks.</p>
  
  <p>Things that <strong>didn't</strong> work:</p>
  
  <ul>
  <li>Sending SIGHUP to the main process: this caused Celery to attempt to "restart," by doing a warm shutdown and then relaunching itself.  Not only does this take a long time, it doesn't even work, because apparently the new process launches before the old one dies, so the new one complains <code>ERROR: Pidfile (/var/run/celeryd.pid) already exists. Seems we're already running? (PID: 13214)</code> and dies immediately.  (This looks like a bug in Celery itself; I've <a href="https://github.com/celery/celery/issues/26" rel="noreferrer">let them know</a> about it.)</li>
  <li>Sending SIGTERM to the main process and then immediately launching a new instance: same issue with the Pidfile.</li>
  <li>Disabling the Pidfile entirely: without it, we have no way of telling which of the 30 Celery process are the main process that needs to be sent a SIGTERM when we want it to do a warm shutdown.  We also have no reliable way to check if the main process is still alive.</li>
  </ul>