title: celery daemon stopped working after I killed celery process
body: <p>I have started job with wrong parameters which would cause it to run forever. I just killed main celery process with kill -9. After that service does not start whatever I do. When running from command line, it starts fine. I'm running Ubuntu 18.04. Log file created but has 0 bytes.</p>
  
  <p>Here is the output of systemctl status celery:</p>
  
  <pre><code>user@ubuntu18-0:~/repository/backend$ systemctl status celery
  ● celery.service - Celery Service
     Loaded: loaded (/lib/systemd/system/celery.service; disabled; vendor preset: enabled)
     Active: inactive (dead)
  
  Mar 12 09:00:10 ubuntu18-0 sh[5889]: celery multi v4.3.0 (rhubarb)
  Mar 12 09:00:10 ubuntu18-0 sh[5889]: &gt; ubuntu18-0@ubuntu18-0: DOWN
  Mar 12 09:40:51 ubuntu18-0 systemd[1]: Starting Celery Service...
  Mar 12 09:40:54 ubuntu18-0 sh[9234]: 2020-03-12 09:40:54,234 - Asynchronous - INFO - ######################## NEW LOG ########################
  Mar 12 09:40:54 ubuntu18-0 sh[9234]: celery multi v4.3.0 (rhubarb)
  Mar 12 09:40:54 ubuntu18-0 sh[9234]: &gt; Starting nodes...
  Mar 12 09:40:54 ubuntu18-0 sh[9234]:         &gt; ubuntu18-0@ubuntu18-0: OK
  Mar 12 09:40:54 ubuntu18-0 systemd[1]: Started Celery Service.
  Mar 12 09:40:55 ubuntu18-0 sh[9266]: celery multi v4.3.0 (rhubarb)
  Mar 12 09:40:55 ubuntu18-0 sh[9266]: &gt; ubuntu18-0@ubuntu18-0: DOWN
  user@ubuntu18-0:~/repository/backend$
  </code></pre>
  
  <p>Here is the output of starting it manually:</p>
  
  <pre><code>./start_celery.sh
  celery multi v4.3.0 (rhubarb)
  &gt; Starting nodes...
  2020-03-12 09:44:12,459 - Asynchronous - INFO - ######################## NEW LOG ########################
          &gt; ubuntu18-0@ubuntu18-0: OK
  </code></pre>
  
  <p>Here is my celery.service:</p>
  
  <pre><code>[Unit]
  Description=Celery Service
  After=network.target
  
  [Service]
  Type=forking
  EnvironmentFile=/etc/celery.conf
  WorkingDirectory=/home/user/repository/backend
  ExecStart=/bin/sh -c '${CELERY_BIN} multi start ${CELERYD_NODES} -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE}  --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}'
  ExecStop=/bin/sh -c '${CELERY_BIN} multi stopwait ${CELERYD_NODES} --pidfile=${CELERYD_PID_FILE}'
  ExecReload=/bin/sh -c '${CELERY_BIN} multi restart ${CELERYD_NODES} -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}'
  Restart=no
  #RestartSec=60
  
  [Install]
  WantedBy=multi-user.target
  </code></pre>
  
  <p>Here is /etc/celery.conf:</p>
  
  <pre><code># Names of node, adapt depending on host
  CELERYD_NODES="ubuntu18-0"
  
  ENV_PYTHON="/home/user/venv/bin/python3"
  
  # Absolute or relative path to the 'celery' command:
  CELERY_BIN="/home/user/venv/bin/celery"
  
  # App instance to use
  # comment out this line if you don't use an app
  CELERY_APP="asynchronous.async_tasks.celery"
  
  # Where to chdir at start.
  CELERYD_CHDIR="/home/user/repository/backend/"
  
  # Extra command-line arguments to the worker
  CELERYD_OPTS="--concurrency=4 -Ofair -Q hcs_async,celery"
  
  # Set logging level to DEBUG
  CELERYD_LOG_LEVEL="INFO"
  
  # %n will be replaced with the first part of the nodename.
  CELERYD_LOG_FILE="/home/user/celery/%n%I.log"
  CELERYD_PID_FILE="/home/user/celery/%n.pid"
  
  # If enabled pid and log directories will be created if missing,
  # and owned by the userid/group configured.
  CELERY_CREATE_DIRS=1
  </code></pre>
  
  <p>Reboot did not help. I suspect there might be a file somewhere which tells systemd or celery that service is in some bad state.</p>