<p dir="auto">Celery 4.0.0<br>
Python 3.5.2<br>
(rest of report output below)</p>
<p dir="auto">I recently pushed celery beat to production (I had been using celery without beat with no issues for several months). It had 1 task to run several times daily, but it duplicated the task many times per second, gradually overwhelming the capacity of my small elasticache instance and resulting in <strong>OOM error</strong>.</p>
<p dir="auto">I was searching for similar errors online and found the following:<br>
<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6489776" data-permission-text="Title is private" data-url="https://github.com/celery/celery/issues/943" data-hovercard-type="issue" data-hovercard-url="/celery/celery/issues/943/hovercard?comment_id=8112993&amp;comment_type=issue_comment" href="https://github.com/celery/celery/issues/943#issuecomment-8112993">#943 (comment)</a></p>
<p dir="auto">While not exactly the same, changing timezone back to default settings seemed to solve the issue  for now. To be clear, previously I had:<br>
enable_utc = False<br>
timezone = 'America/New_York'</p>
<p dir="auto">I changed these to:<br>
enable_utc = True</p>
<p dir="auto">and it seemed to solve the problem (for now).</p>
<p dir="auto">** celery -A [proj] report:</p>
<p dir="auto">software -&gt; celery:4.0.0 (latentcall) kombu:4.0.0 py:3.5.2<br>
billiard:3.5.0.2 redis:2.10.5<br>
platform -&gt; system:Linux arch:64bit, ELF imp:CPython<br>
loader   -&gt; celery.loaders.app.AppLoader<br>
settings -&gt; transport:redis results:redis://[elasticache]</p>
<p dir="auto">result_persistent: False<br>
enable_utc: False<br>
result_serializer: 'json'<br>
include: [tasks -- redacted]<br>
result_backend: 'redis://[elasticache]<br>
task_create_missing_queues: True<br>
task_acks_late: False<br>
timezone: 'America/New_York'<br>
broker_url: '[elasticache]'<br>
task_serializer: 'json'<br>
crontab: &lt;class 'celery.schedules.crontab'&gt;<br>
broker_transport_options: {<br>
'visibility_timeout': 600}<br>
task_time_limit: 200<br>
task_always_eager: False<br>
task_queues:<br>
(&lt;unbound Queue default -&gt; &lt;unbound Exchange default(direct)&gt; -&gt; default&gt;,<br>
&lt;unbound Queue render -&gt; &lt;unbound Exchange media(direct)&gt; -&gt; media.render&gt;,<br>
&lt;unbound Queue emails -&gt; &lt;unbound Exchange media(direct)&gt; -&gt; media.emails&gt;,<br>
&lt;unbound Queue other -&gt; &lt;unbound Exchange media(direct)&gt; -&gt; media.other&gt;)<br>
beat_schedule: {<br>
'lookup-emails-to-send': {   'args': [],<br>
'schedule': &lt;crontab: 0 9,12,15,18,21 * * * (m/h/d/dM/MY)&gt;,<br>
'task': 'send_emails'}}<br>
accept_content: ['json']<br>
task_default_exchange_type: 'direct'<br>
task_routes:<br>
('worker.routes.AppRouter',)<br>
worker_prefetch_multiplier: 1<br>
BROKER_URL: [elasticache]</p>
<p dir="auto">** Worker error message after OOM:<br>
Traceback (most recent call last):<br>
File "python3.5/site-packages/celery/beat.py", line 299, in apply_async<br>
**entry.options)<br>
File "python3.5/site-packages/celery/app/task.py", line 536, in apply_async<br>
**options<br>
File "python3.5/site-packages/celery/app/base.py", line 717, in send_task<br>
amqp.send_task_message(P, name, message, **options)<br>
File "python3.5/site-packages/celery/app/amqp.py", line 554, in send_task_message<br>
**properties<br>
File "python3.5/site-packages/kombu/messaging.py", line 178, in publish<br>
exchange_name, declare,<br>
File "python3.5/site-packages/kombu/connection.py", line 527, in _ensured<br>
errback and errback(exc, 0)<br>
File "python3.5/contextlib.py", line 77, in <strong>exit</strong><br>
self.gen.throw(type, value, traceback)<br>
File "python3.5/site-packages/kombu/connection.py", line 419, in _reraise_as_library_errors<br>
sys.exc_info()[2])<br>
File "python3.5/site-packages/vine/five.py", line 175, in reraise<br>
raise value.with_traceback(tb)<br>
File "python3.5/site-packages/kombu/connection.py", line 414, in _reraise_as_library_errors<br>
yield<br>
File "python3.5/site-packages/kombu/connection.py", line 494, in _ensured<br>
return fun(*args, **kwargs)<br>
File "python3.5/site-packages/kombu/messaging.py", line 200, in _publish<br>
mandatory=mandatory, immediate=immediate,<br>
File "python3.5/site-packages/kombu/transport/virtual/base.py", line 608, in basic_publish<br>
return self._put(routing_key, message, **kwargs)<br>
File "python3.5/site-packages/kombu/transport/redis.py", line 766, in _put<br>
client.lpush(self._q_for_pri(queue, pri), dumps(message))<br>
File "python3.5/site-packages/redis/client.py", line 1227, in lpush<br>
return self.execute_command('LPUSH', name, *values)<br>
File "python3.5/site-packages/redis/client.py", line 573, in execute_command<br>
return self.parse_response(connection, command_name, **options)<br>
File "python3.5/site-packages/redis/client.py", line 585, in parse_response<br>
response = connection.read_response()<br>
File "python3.5/site-packages/redis/connection.py", line 582, in read_response<br>
raise response<br>
kombu.exceptions.OperationalError: OOM command not allowed when used memory &gt; 'maxmemory'.</p>
<p dir="auto">** Example log showing rapidly duplicated beat tasks:<br>
[2017-05-20 09:00:00,096: INFO/PoolWorker-1] Task send_emails[1d24250f-c254-4fd8-8ffc-cfb17b98a392] succeeded in 0.01704882364720106s: True<br>
[2017-05-20 09:00:00,098: INFO/MainProcess] Received task: send_emails[e968cc65-0924-4d31-b2d7-9a3e3f5aefda]<br>
[2017-05-20 09:00:00,115: INFO/PoolWorker-1] Task send_emails[e968cc65-0823-4d31-b2d7-9a3e3f5aefda] succeeded in 0.015948095358908176s: True<br>
[2017-05-20 09:00:00,117: INFO/MainProcess] Received task: send_emails[2fb9f7ac-5f6a-42e1-813e-25ea4023dc81]<br>
[2017-05-20 09:00:00,136: INFO/PoolWorker-1] Task send_emails[2fb9f7ac-5f6a-42e1-813e-25ea4023dc81] succeeded in 0.018534105271100998s: True<br>
[2017-05-20 09:00:00,140: INFO/MainProcess] Received task: send_emails[59493c0b-1858-497f-a0dc-a4c8e4ba3a63]<br>
[2017-05-20 09:00:00,161: INFO/PoolWorker-1] Task send_emails[59493c0b-1858-497f-a0dc-a4c8e4ba3a63] succeeded in 0.020539570599794388s: True<br>
[2017-05-20 09:00:00,164: INFO/MainProcess] Received task: send_emails[3aac612f-75dd-4530-9b55-1e288bec8db4]<br>
[2017-05-20 09:00:00,205: INFO/PoolWorker-1] Task send_emails[3aac612f-75dd-4530-9b55-1e288bec8db4] succeeded in 0.04063933063298464s: True<br>
[2017-05-20 09:00:00,208: INFO/MainProcess] Received task: send_emails[7edf60cb-d1c4-4ae5-af10-03414da672fa]</p>
<h2 dir="auto">I happened to install celery 4.1.0 with django_celey_beat 1.0.1, the DatabaseScheduler seems not working well.</h2>
<p dir="auto">[2017-08-07 21:12:10,790: DEBUG/MainProcess] DatabaseScheduler: Fetching database schedule<br>
[2017-08-07 21:12:10,797: DEBUG/MainProcess] Current schedule:<br>
&lt;ModelEntry: celery.backend_cleanup celery.backend_cleanup(<em>[], **{}) &lt;crontab: 0 4 * * * (m/h/d/dM/MY)&gt;&gt;<br>
&lt;ModelEntry: schedule GeneBank.tasks.test(</em>[], **{}) &lt;crontab: * * * * * (m/h/d/dM/MY)&gt;&gt;<br>
[2017-08-07 21:12:10,807: DEBUG/MainProcess] beat: Ticking with max interval-&gt;5.00 seconds<br>
[2017-08-07 21:12:10,809: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.<br>
[2017-08-07 21:12:15,813: DEBUG/MainProcess] beat: Synchronizing schedule...<br>
[2017-08-07 21:12:15,813: INFO/MainProcess] Writing entries...<br>
[2017-08-07 21:12:15,816: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.<br>
[2017-08-07 21:12:20,818: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.<br>
[2017-08-07 21:12:25,825: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.<br>
[2017-08-07 21:12:30,831: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.<br>
[2017-08-07 21:12:35,839: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.<br>
[2017-08-07 21:12:40,844: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.<br>
[2017-08-07 21:12:45,851: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.<br>
[2017-08-07 21:12:50,854: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.<br>
[2017-08-07 21:12:55,860: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.<br>
[2017-08-07 21:13:00,862: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.<br>
[2017-08-07 21:13:05,870: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.<br>
^C[2017-08-07 21:13:10,245: INFO/MainProcess] Writing entries...<br>
[2017-08-07 21:13:10,246: INFO/MainProcess] Writing entries...</p>
<h2 dir="auto">As u can see, the scheduler was supposed to send beat at every minute, but the beat didn't show up,  (i've set crontab to be all *, so couldn't be timezone problem)</h2>
<h1 dir="auto">But in celery 4.0.2, everything goes fine! I don't know whether it is a bug or not. Maybe the django_celery_beat isn't compatible with 4.1.0.</h1>
<p dir="auto">[2017-08-07 21:18:43,339: DEBUG/MainProcess] Current schedule:<br>
&lt;ModelEntry: celery.backend_cleanup celery.backend_cleanup(<em>[], **{}) &lt;crontab: 0 4 * * * (m/h/d/dM/MY)&gt;&gt;<br>
&lt;ModelEntry: schedule GeneBank.tasks.test(</em>[], **{}) &lt;crontab: * * * * * (m/h/d/dM/MY)&gt;&gt;<br>
[2017-08-07 21:18:43,351: DEBUG/MainProcess] beat: Ticking with max interval-&gt;5.00 seconds<br>
[2017-08-07 21:18:43,364: INFO/MainProcess] Scheduler: Sending due task schedule (GeneBank.tasks.test)<br>
[2017-08-07 21:18:43,376: DEBUG/MainProcess] beat: Synchronizing schedule...<br>
[2017-08-07 21:18:43,376: INFO/MainProcess] Writing entries...<br>
[2017-08-07 21:18:43,380: DEBUG/MainProcess] GeneBank.tasks.test sent. id-&gt;9c1bdf10-0a5f-440a-98db-9eb24433a8d4<br>
[2017-08-07 21:18:43,381: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.<br>
[2017-08-07 21:18:48,386: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.<br>
[2017-08-07 21:18:53,392: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.<br>
[2017-08-07 21:18:58,397: DEBUG/MainProcess] beat: Waking up in 1.59 seconds.<br>
[2017-08-07 21:19:00,001: INFO/MainProcess] Scheduler: Sending due task schedule (GeneBank.tasks.test)</p>