<p dir="auto">Hi,<br>
I am using celery as backend asynchronous task processing, I am facing issue as celery workers stops picking tasks after few minutes ~5-10 min.<br>
I tried scalling up workers, routing tasks to queues etc but nothing helps me out. below is my settings.</p>
<pre class="notranslate">BROKER_URL = 'amqp://****@fast-polar-bear.rmq.cloudamqp.com/***'
CELERY_DEFAULT_QUEUE = 'proj.env.defaultnew'
CELERY_QUEUES = (
    Queue(
        'proj.env.defaultnew',
        Exchange('proj.env.defaultnew'),
        routing_key='proj.env.defaultnew'),)
CELERY_DEFAULT_EXCHANGE_TYPE = 'direct'
CELERY_ROUTES = {
    # High priority tasks maping
    'datasync.tasks.api_push_ship_status': {
        'queue': 'proj.env.priority_queue',
    },
    'datasync.tasks.api_push_pack_status': {
        'queue': 'proj.env.priority_queue',
    },
    'gmflow.tasks.post_activity_call': {
        'queue': 'proj.env.priority_queue',
    },
    'ims.tasks.create_item_history': {
        'queue': 'proj.env.priority_queue',
    },
    'remarks.tasks.insert_remaks_task': {
        'queue': 'proj.env.priority_queue',
    },
    'ims.tasks.sync_items_task': {
        'queue': 'proj.env.priority_queue',
    },
    'datasync.tasks.file_datasync_task': {
        'queue': 'proj.env.priority_queue',
    },
    # Low priority tasks maping
    'wms.tasks.queue_report': {
        'queue': 'proj.env.low_priority',
    },
    'oms.tasks.update_order_status_task': {
        'queue': 'proj.env.low_priority',
    },
    'datasync.tasks.datasync_task': {
        'queue': 'proj.env.low_priority',
    },
}
</pre>
<p dir="auto">Number of workers per queue<br>
proj.env.low_priority - 10 workers - time consuming tasks with low frequency<br>
proj.env.default - 10 workers - normal tasks normal frequency<br>
proj.env.priority_queue - 10 workers - very less time consuming high frequency</p>
<p dir="auto">Consumer settings</p>
<pre class="notranslate">--without-gossip
--without-heartbeat
--without-mingle
--loglevel=info
</pre>
<p dir="auto">Celery versions</p>
<pre class="notranslate">django1.8.1
celery==3.1.18
django-celery==3.1.10
amqp==1.4.7
kombu==3.0.26
</pre>
<p dir="auto">Consumer Command</p>
<pre class="notranslate">python manage.py celery worker -n priority_queue:%n -Q proj.env.priority_queue --settings=settings.prod  --without-gossip --without-heartbeat --without-mingle --loglevel=info
</pre>
<p dir="auto">please help and suggest the changes if required</p>
<p dir="auto">See test case here:<br>
<a href="https://github.com/sabw8217/celery_test">https://github.com/sabw8217/celery_test</a></p>
<p dir="auto">If I enqueue a number of tasks, and then start a worker with --without-mingle and --without-gossip, the worker seems to hang and not actually run any of the tasks I enqueued. I'm running this on debian. On my dev box I have pretty consistently been able to cause the worker to "wake up" and start consuming tasks by issuing a 'celery inspect' command. 'celery inspect active' will return something like:<br>
-&gt; <a href="mailto:celery@aaron-dev.localdomain">celery@aaron-dev.localdomain</a>: OK<br>
- empty -<br>
And then worker the will go through the tasks.</p>
<p dir="auto">Debug output from the worker looks like:<br>
[2014-02-04 13:53:47,628: DEBUG/MainProcess] | Worker: Starting Hub<br>
[2014-02-04 13:53:47,628: DEBUG/MainProcess] ^-- substep ok<br>
[2014-02-04 13:53:47,628: DEBUG/MainProcess] | Worker: Starting Pool<br>
[2014-02-04 13:53:47,633: DEBUG/MainProcess] ^-- substep ok<br>
[2014-02-04 13:53:47,634: DEBUG/MainProcess] | Worker: Starting Consumer<br>
[2014-02-04 13:53:47,635: DEBUG/MainProcess] | Consumer: Starting Connection<br>
[2014-02-04 13:53:47,644: INFO/MainProcess] Connected to amqp://guest@localhost:5672//<br>
[2014-02-04 13:53:47,645: DEBUG/MainProcess] ^-- substep ok<br>
[2014-02-04 13:53:47,645: DEBUG/MainProcess] | Consumer: Starting Events<br>
[2014-02-04 13:53:47,659: DEBUG/MainProcess] ^-- substep ok<br>
[2014-02-04 13:53:47,661: DEBUG/MainProcess] | Consumer: Starting Heart<br>
[2014-02-04 13:53:47,663: DEBUG/MainProcess] ^-- substep ok<br>
[2014-02-04 13:53:47,663: DEBUG/MainProcess] | Consumer: Starting Tasks<br>
[2014-02-04 13:53:47,666: DEBUG/MainProcess] basic.qos: prefetch_count-&gt;4<br>
[2014-02-04 13:53:47,667: DEBUG/MainProcess] ^-- substep ok<br>
[2014-02-04 13:53:47,667: DEBUG/MainProcess] | Consumer: Starting Control<br>
[2014-02-04 13:53:47,669: DEBUG/MainProcess] ^-- substep ok<br>
[2014-02-04 13:53:47,670: DEBUG/MainProcess] | Consumer: Starting event loop<br>
[2014-02-04 13:53:47,670: WARNING/MainProcess] <a href="mailto:celery@aaron-dev.localdomain">celery@aaron-dev.localdomain</a> ready.<br>
[2014-02-04 13:53:47,670: DEBUG/MainProcess] | Worker: Hub.register Pool...</p>