Celery (Django) ImportError: No module named billiard.exceptions
Question: I'm using current versions (Celery 3.1.19, Billiard 3.3.0.21) and unable to
successfully deploy my Django site to AWS Elastic Beanstalk.

I'm trying to figure out if this is a WSGI issue, a system path issue, an app
issue, etc. I had the site working, but I upgraded Celery and other pip
packages as well as Amazon's AMI (at the same time... not smart for
debugging).

This is the stack trace:

    
    
    Target WSGI script '/opt/python/current/app/mysite/wsgi.py' cannot be loaded as Python module.
    Exception occurred processing WSGI script '/opt/python/current/app/mysite/wsgi.py'.
    Traceback (most recent call last):
    File "/opt/python/current/app/mysite/wsgi.py", line 23, in <module>
        application = get_wsgi_application()
    File "/opt/python/run/venv/lib/python2.7/site-packages/django/core/wsgi.py", line 14, in get_wsgi_application
        django.setup()
    File "/opt/python/run/venv/lib/python2.7/site-packages/django/__init__.py", line 17, in setup
        configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
    File "/opt/python/run/venv/lib/python2.7/site-packages/django/conf/__init__.py", line 48, in __getattr__
        self._setup(name)
    File "/opt/python/run/venv/lib/python2.7/site-packages/django/conf/__init__.py", line 44, in _setup
        self._wrapped = Settings(settings_module)
    File "/opt/python/run/venv/lib/python2.7/site-packages/django/conf/__init__.py", line 92, in __init__
        mod = importlib.import_module(self.SETTINGS_MODULE)
    File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
        __import__(name)
    File "/opt/python/current/app/mysite/__init__.py", line 5, in <module>
        from .celery import app as celery_app
    File "/opt/python/current/app/mysite/celery.py", line 3, in <module>
        from celery import Celery
    File "/opt/python/run/venv/lib/python2.7/site-packages/celery/five.py", line 306, in __getattr__
        module = __import__(self._object_origins[name], None, None, [name])
    File "/opt/python/run/venv/lib/python2.7/site-packages/celery/app/__init__.py", line 14, in <module>
        from celery import _state
    File "/opt/python/run/venv/lib/python2.7/site-packages/celery/_state.py", line 20, in <module>
        from celery.utils.threads import LocalStack
    File "/opt/python/run/venv/lib/python2.7/site-packages/celery/utils/__init__.py", line 27, in <module>
        from celery.exceptions import CPendingDeprecationWarning, CDeprecationWarning
    File "/opt/python/run/venv/lib/python2.7/site-packages/celery/exceptions.py", line 15, in <module>
        from billiard.exceptions import (  # noqa
    ImportError: No module named billiard.exceptions
    AH00124: Request exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace.
    

Just a line before `application = get_wsgi_application()` I print `sys.path`:

    
    
    ['/opt/python/run/venv/lib/python2.7/site-packages',
    '/opt/python/current/app',
    '',
    '/opt/python/run/baselinenv/local/lib64/python2.7/site-packages', 
    '/opt/python/run/baselinenv/local/lib/python2.7/site-packages', 
    '/opt/python/run/baselinenv/lib64/python2.7', 
    '/opt/python/run/baselinenv/lib/python2.7', 
    '/opt/python/run/baselinenv/lib64/python2.7/site-packages', 
    '/opt/python/run/baselinenv/lib/python2.7/site-packages', 
    '/opt/python/run/baselinenv/lib64/python2.7/lib-dynload', 
    '/usr/lib64/python2.7', 
    '/usr/lib/python2.7']
    

In my wsgi.conf:

    
    
    WSGIPythonHome /opt/python/run/baselinenv
    

Any guidance on even where to start debugging this will be helpful.


Answer: The problem was my `WSGIDaemonProcess` `python-path` in my wsgi.conf. I had to
add a reference to `lib64` in addition to the normal `lib` directory.

    
    
    python-path=/opt/python/current/app:/opt/python/run/venv/lib/python2.7/site-packages:/opt/python/run/venv/lib64/python2.7/site-packages
    

