Django with wsgi sporadically fails with "Premature end of script headers:"
Question: I have a site that is running for months now on Apache2 and periodically it
will just hiccup with the following:

    
    
    [Sat Nov 12 06:18:34 2011] [error] [client X.Y.Z.158] Premature end of script headers: sleepsoundly_wsgi.py
    [Sat Nov 12 06:18:49 2011] [error] [client X.Y.Z.158] Premature end of script headers: sleepsoundly_wsgi.py
    

It has run 1000s of requests without a problem, but periodically it will do
this a couple of times and then everything will be ok. This is happening while
uploading about 300 files (.5 MB each). Each file is upload separately, 3
files at a time, 225 files upload fine, 226 and 227 failed, and then 228 ->
end all worked normally. It doesn't do this every time, just occasionally and
it isn't always these files that fail. Another time file #291 failed and all
the rest worked.

I've got nothing in the log to go on other than this cryptic message.

I've checked and the only version of python on the machine is 2.7.1. I don't
get an email from django, I don't get any of the normal clues as to what might
be happening. I'm curious how to start troubleshooting this one. It recovers
on its own, the automated program uploading the files keeps moving. How do I
figure out what is going on in this case?

    
    
    Server version: Apache/2.2.17 (Ubuntu)
    Server built:   Sep  1 2011 09:25:26
    mod_wsgi: Version: 3.3-2ubuntu2
    
    Server MPM:     Prefork
      threaded:     no
        forked:     yes (variable process count)
    
    wsgi.conf has no lines in it that are not commented out.
    
    VirtualHost setup:
        WSGIDaemonProcess myemr user=mjones processes=1 maximum-requests=500 threads=15
        WSGIProcessGroup  myemr
        WSGIScriptAlias   / /var/www/Python/myemr/myemr/deploy/myemr_wsgi.py
    
    myemr_wsgi.py
        from os.path import abspath, dirname, join
        import sys
    
        # For packages that don't play well with mod_wsgi
        sys.stdout = sys.stderr
    
        sys.path.insert(0, abspath(join(dirname(__file__), "../..")))
        sys.path.insert(0, abspath(join(dirname(__file__), "../../myemr")))
        sys.path.insert(0, abspath(join(dirname(__file__), "../../myemr/apps")))
        sys.path.insert(0, abspath(join(dirname(__file__), "../../lib/python2.7/site-packages/")))
    
        # We have to add both of these because they are  installed with git?
        sys.path.insert(0, abspath(join(dirname(__file__), "../../src/pinax/")))
    
        from django.core.handlers.wsgi import WSGIHandler
        import pinax.env
    
        # setup the environment for Django and Pinax
        pinax.env.setup_environ(project_path='myemr')
    
        # set application for WSGI processing
        application = WSGIHandler()
    


Answer: The problem is likely because you have 'maximum-requests' set to 500. This
will cause the mod_wsgi daemon processes to be restarted periodically. Since
the mod_wsgi daemon processes are running in multithreaded configuration, if
there was a stuck request, or long running request which doesn't finish before
the forced shutdown timeout when the restart is being done, then it will be
aborted by virtue of the process restart. The Apache child worker process will
then see that as a request terminated without returning headers with the
message you are seeing.

Moral of the story, DO NOT use 'maximum-requests' options in production
systems unless you have a very good reason such as out of control memory
growth.

Version 4.0 of mod_wsgi will have an optional but slightly more graceful
restart option which can be applied for this case, but it still can't wait
forever and a stuck request will still have to be aborted at some point and
you will still see the message.

BTW, don't specify 'processes=1' as it defaults to one process anyway and any
use of the 'processes' option, even with value '1', causes 'wsgi.multiprocess'
to be set True. Use of 'processes=1' should only be done where have a single
process AND you have many Apache instances which you are load balancing
across, also with a single process.

