title: Celery tasks almost immediately get "stuck" when using Celery Beat and countdown with .apply_async()
body: <p>Here is <a href="https://github.com/synthead/pidrator/blob/testing/pidrator/hardware_controller.py" rel="nofollow">my Celery "tasks" file</a> and <a href="https://github.com/synthead/pidrator/blob/testing/pidrator/settings.py" rel="nofollow">my Django project's settings.py</a>, which has this Celery Beat configuration (the timedelta is set to every 3 seconds in a user-editable configuration file):</p>
  
  <pre><code>CELERYBEAT_SCHEDULE = {
    "UpdateEnabledSensors": {
      "task": "pidrator.hardware_controller.UpdateEnabledSensors",
      "schedule": timedelta(seconds=PIDRATOR_SENSOR_UPDATE_FREQUENCY)
    }
  }
  </code></pre>
  
  <p>When starting the Celery workers via <code>celery -A pidrator -B -l info</code>, I get <a href="http://pastie.org/pastes/9278664/text" rel="nofollow">this output</a>.  </p>
  
  <p>As you can see, the tasks just kinda start, well, slowly dying off.  First, ActuateRelay stops responding to requests, then WateringCycle, then UpdateSensor, etc.  What's even more bizarre is that there are no errors reported via Celery.</p>
  
  <p>I think the problem is in the <code>WateringCycle</code> task, which looks like this:</p>
  
  <pre><code>@celery_pidrator.task
  def WateringCycle(irrigator):
    from celery import chain
    chain(
        SetWateringCycle.si(irrigator, True),
        ActuateRelay.si(irrigator.relay, True),
        ActuateRelay.subtask(
            (irrigator.relay, False),
            countdown=irrigator.watering_seconds,
            immutable=True),
        SetWateringCycle.subtask(
            (irrigator, False),
            countdown=irrigator.timeout_seconds,
            immutable=True)
    ).apply_async()
  </code></pre>
  
  <p>When I comment out its contents and put <code>logger.warning("test")</code> in it instead, the celery tasks seem to finish and the queue doesn't get "stuck."  I don't think it's because of the chain, however, because if I replace the task's contents with:</p>
  
  <pre><code>@celery_pidrator.task
  def WateringCycle(irrigator):
    SetWateringCycle(irrigator, True)
    ActuateRelay(irrigator.relay, True)
    ActuateRelay.apply_async(
        (irrigator.relay, False),
        countdown=irrigator.watering_seconds)
    SetWateringCycle.apply_async(
        (irrigator, False),
        countdown=irrigator.watering_seconds + irrigator.timeout_seconds)
  </code></pre>
  
  <p>...it still gets "stuck."  I don't really understand how the task queues and workers work outside that they live in pools and process tasks, but something's telling me that the tasks with a countdown are "filling up the queues" (pardon my ignorance in semantics).  However, if I add <code>--autoscale=50,1</code> to celery, when it gets "stuck" (I really wish I had a better term for this), celery workers just accumulate until all 50 workers are present, and it still doesn't process any tasks.</p>
  
  <p>I'm using Python 3, Redis, and mysql.connector with Celery.  I've tried using RabbitMQ and even porting the entire project to Python 2.x to use librabbitmq, but it still behaves the same way. Here are some package versions if it helps:</p>
  
  <pre><code>python 3.4.1-1
  python-celery 3.1.12-1
  python2-kombu 3.0.19-1
  python-billiard 3.3.0.17-1
  python-django 1.6.5-1
  python-redis 2.10.1-1
  redis 2.8.10-1
  python-mysql-connector 1.2.2-1
  </code></pre>