Gunicorn Django [CRITICAL] WORKER TIMEOUT
Question: Since I did a pip install google-api-python-client I have my Gunicorn workers
stoping after timeout.

Django==1.5.3

Gunicorn==0.12.2

I'm not really sure if it comes from the pip but I did nothing particular
except a database migration which migrated without error.

I use this command for Gunicorn:

    
    
    gunicorn_django myapp.py --bind 127.0.0.1:8181 --timeout 120 --log-file /tmp/myapp.gunicorn.log --log-level info --workers 8 --pid /tmp/myapp.pid
    

I tryed the param --spew to have some trace but it doesn't help me:

    
    
    [2016-06-13 21:09:52 +0000] [15602] [INFO] Worker exiting (pid: 15602)
    [2016-06-13 21:09:52 +0000] [15601] [ERROR] Exception in worker process
    Traceback (most recent call last):
      File "/home/myapp/.local/share/virtualenvs/myapp/lib/python2.7/site-packages/gunicorn/arbiter.py", line 557, in spawn_worker
        worker.init_process()
      File "/home/myapp/.local/share/virtualenvs/myapp/lib/python2.7/site-packages/gunicorn/workers/base.py", line 126, in init_process
        self.load_wsgi()
      File "/home/myapp/.local/share/virtualenvs/myapp/lib/python2.7/site-packages/gunicorn/workers/base.py", line 136, in load_wsgi
        self.wsgi = self.app.wsgi()
      File "/home/myapp/.local/share/virtualenvs/myapp/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
        self.callable = self.load()
      File "/home/myapp/.local/share/virtualenvs/myapp/lib/python2.7/site-packages/gunicorn/app/djangoapp.py", line 106, in load
        return mod.make_wsgi_application()
      File "/home/myapp/.local/share/virtualenvs/myapp/lib/python2.7/site-packages/gunicorn/app/django_wsgi.py", line 37, in make_wsgi_application
        if get_validation_errors(s):
      File "/home/myapp/.local/share/virtualenvs/myapp/lib/python2.7/site-packages/django/core/management/validation.py", line 35, in get_validation_errors
        for (app_name, error) in get_app_errors().items():
      File "/home/myapp/.local/share/virtualenvs/myapp/lib/python2.7/site-packages/django/db/models/loading.py", line 166, in get_app_errors
        self._populate()
      File "/home/myapp/.local/share/virtualenvs/myapp/lib/python2.7/site-packages/django/db/models/loading.py", line 72, in _populate
        self.load_app(app_name, True)
      File "/home/myapp/.local/share/virtualenvs/myapp/lib/python2.7/site-packages/django/db/models/loading.py", line 96, in load_app
        models = import_module('.models', app_name)
      File "/home/myapp/.local/share/virtualenvs/myapp/lib/python2.7/site-packages/django/utils/importlib.py", line 35, in import_module
        __import__(name)
      File "/home/myapp/prod/apps/admin/models.py", line 5, in <module>
        from django.contrib.auth.models import User
      File "/home/myapp/.local/share/virtualenvs/myapp/lib/python2.7/site-packages/django/contrib/auth/models.py", line 18, in <module>
        from django.contrib.auth.hashers import (
      File "/home/myapp/.local/share/virtualenvs/myapp/lib/python2.7/site-packages/django/contrib/auth/hashers.py", line 8, in <module>
        from django.test.signals import setting_changed
      File "/home/myapp/.local/share/virtualenvs/myapp/lib/python2.7/site-packages/django/test/__init__.py", line 6, in <module>
        from django.test.testcases import (TestCase, TransactionTestCase,
      File "/home/myapp/.local/share/virtualenvs/myapp/lib/python2.7/site-packages/django/test/testcases.py", line 35, in <module>
        from django.test import _doctest as doctest
      File "/home/myapp/.local/share/virtualenvs/myapp/lib/python2.7/site-packages/django/test/_doctest.py", line 104, in <module>
        import unittest, difflib, pdb, tempfile
      File "/home/myapp/.local/share/virtualenvs/myapp/lib/python2.7/site-packages/pdbpp-0.7.2-py2.7.egg/pdb.py", line 38, in <module>
        pdb = import_from_stdlib('pdb')
      File "/home/myapp/.local/share/virtualenvs/myapp/lib/python2.7/site-packages/pdbpp-0.7.2-py2.7.egg/pdb.py", line 35, in import_from_stdlib
        mydict = execfile(pyfile, result.__dict__)
      File "/usr/local/lib/python2.7/pdb.py", line 3, in <module>
        """A Python debugger."""
      File "/usr/local/lib/python2.7/pdb.py", line 3, in <module>
        """A Python debugger."""
      File "/home/myapp/.local/share/virtualenvs/myapp/lib/python2.7/site-packages/gunicorn/debug.py", line 40, in __call__
        line = src[lineno]
    IndexError: tuple index out of range
    [2016-06-13 21:09:52 +0000] [15601] [INFO] Worker exiting (pid: 15601)
    

As the problem came in the same time I installed google api client, I suspect
pip to have upgraded some libs that are not compatible with my gunicorn or
Django. I checked the pip log without success also.

If I run my Django app with runserver I can't see any bug, it seems very
related to Gunicorn.

Is there a deeper way to debug Gunicorn ?


Answer: After struggling hours I finally found a clue in the pip log
(HOME/.pip/pip.log) .

Installing google api client upgraded some of my previous libs like these:

    
    
    Installing collected packages: pyopenssl, six, cryptography, idna, pyasn1, setuptools, enum34, ipaddress, cffi, pycparser
      Found existing installation: pyOpenSSL 0.14
        Uninstalling pyOpenSSL:
    ...
      Found existing installation: six 1.9.0
        Uninstalling six:
    ...
      Found existing installation: cryptography 0.7.1
        Uninstalling cryptography:
    

I noticed also some installing warning for cyptography. I decided to put back
the old libs.

  * pyOpenSSL 0.14
  * six 1.9.0
  * cryptography 0.7.1

And it solved the problem. I don't know if it is pyopenssl or cryptography but
it is getting really boring to have all these libs problems.

Hope this will help someone next time.

