SSL SYSCALL error: Bad file descriptor on Heroku with postgres and Celery
Question: I've been using Celery successfully with a Django site on Heroku but it's just
started generating the error below, which stops it running. It looks like it's
having trouble with postgres, but I'm stumped as to how to fix it, given it's
Celery rather than my code that's having the problem (I assume...).

I'm using CloudAMPQ as a broker, and my Django settings include:

    
    
    CELERYBEAT_SCHEDULER = 'djcelery.schedulers.DatabaseScheduler'
    

Here's the traceback from the Heroku logs:

    
    
    Traceback (most recent call last):
      File "/app/.heroku/python/lib/python3.5/site-packages/kombu/utils/__init__.py", line 323, in __get__
        return obj.__dict__[self.__name__]
    KeyError: 'scheduler'
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/app/.heroku/python/lib/python3.5/site-packages/django/db/backends/utils.py", line 64, in execute
        return self.cursor.execute(sql, params)
    psycopg2.OperationalError: SSL SYSCALL error: Bad file descriptor
    
    
    The above exception was the direct cause of the following exception:
    
    Traceback (most recent call last):
      File "/app/.heroku/python/lib/python3.5/site-packages/billiard/process.py", line 292, in _bootstrap
        self.run()
      File "/app/.heroku/python/lib/python3.5/site-packages/celery/beat.py", line 553, in run
        self.service.start(embedded_process=True)
      File "/app/.heroku/python/lib/python3.5/site-packages/celery/beat.py", line 470, in start
        humanize_seconds(self.scheduler.max_interval))
      File "/app/.heroku/python/lib/python3.5/site-packages/kombu/utils/__init__.py", line 325, in __get__
        value = obj.__dict__[self.__name__] = self.__get(obj)
      File "/app/.heroku/python/lib/python3.5/site-packages/celery/beat.py", line 512, in scheduler
        return self.get_scheduler()
      File "/app/.heroku/python/lib/python3.5/site-packages/celery/beat.py", line 507, in get_scheduler
        lazy=lazy)
      File "/app/.heroku/python/lib/python3.5/site-packages/celery/utils/imports.py", line 53, in instantiate
        return symbol_by_name(name)(*args, **kwargs)
      File "/app/.heroku/python/lib/python3.5/site-packages/djcelery/schedulers.py", line 151, in __init__
        Scheduler.__init__(self, *args, **kwargs)
      File "/app/.heroku/python/lib/python3.5/site-packages/celery/beat.py", line 185, in __init__
        self.setup_schedule()
      File "/app/.heroku/python/lib/python3.5/site-packages/djcelery/schedulers.py", line 158, in setup_schedule
        self.install_default_entries(self.schedule)
      File "/app/.heroku/python/lib/python3.5/site-packages/djcelery/schedulers.py", line 251, in schedule
        self._schedule = self.all_as_schedule()
      File "/app/.heroku/python/lib/python3.5/site-packages/djcelery/schedulers.py", line 164, in all_as_schedule
        for model in self.Model.objects.enabled():
      File "/app/.heroku/python/lib/python3.5/site-packages/django/db/models/query.py", line 258, in __iter__
        self._fetch_all()
      File "/app/.heroku/python/lib/python3.5/site-packages/django/db/models/query.py", line 1074, in _fetch_all
        self._result_cache = list(self.iterator())
      File "/app/.heroku/python/lib/python3.5/site-packages/django/db/models/query.py", line 52, in __iter__
        results = compiler.execute_sql()
      File "/app/.heroku/python/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 848, in execute_sql
        cursor.execute(sql, params)
      File "/app/.heroku/python/lib/python3.5/site-packages/django/db/backends/utils.py", line 64, in execute
        return self.cursor.execute(sql, params)
      File "/app/.heroku/python/lib/python3.5/site-packages/django/db/utils.py", line 95, in __exit__
        six.reraise(dj_exc_type, dj_exc_value, traceback)
      File "/app/.heroku/python/lib/python3.5/site-packages/django/utils/six.py", line 685, in reraise
        raise value.with_traceback(tb)
      File "/app/.heroku/python/lib/python3.5/site-packages/django/db/backends/utils.py", line 64, in execute
        return self.cursor.execute(sql, params)
    django.db.utils.OperationalError: SSL SYSCALL error: Bad file descriptor
    


Answer: I've resolved the issue now... there was a line of my Django code which had
caused an Internal Server Error in the past -- I think, early on in Django
starting up, it was trying to access a model before the migrations that
created the model had run.

I'd resolved that but noticed these "SSL SYSCALL error"s started about the
same time. So I removed that line of code, and Celery has started up again.

It could be coincidence. And I don't understand why this fixed things.

Ideally I'd still like to understand what the error above actually _means_ so
I'd have a better chance of fixing such a thing in the future.

