Proper way to restart HTTP server in Python
Question: I'm writing a HTTP server in Python using the code snippet below. The server
works well until some IOError happens causing it to restart. Something is
wrong with my restart handling since the server starts up fine but does not
accept any requests after that.

Is there anything wrong in this code?

    
    
    #!/bin/python
    from BaseHTTPServer import HTTPServer
    import json
    import config
    import time
    import socket
    from SimpleHTTPServer import SimpleHTTPRequestHandler
    from SocketServer import BaseServer
    
    
    class MyHandler(SimpleHTTPRequestHandler):
       def parseArgs(self):
            args = {}
            try:
                config.logging.info(self.path)
                args_list = self.path.split("?")[1].split("&")
                for entry in args_list:
                   args[entry.split("=")[0]] = entry.split("=")[1]
            except IndexError:
                pass
            return args
    
       def do_GET(self):
          try:
            response = {}
            args = self.parseArgs()
            config.logging.debug("Handle the request")
            self._handle_request(args, response)
            config.logging.debug("Write the header back to the client.")
            self.send_response(200)
            self.send_header('Access-Control-Allow-Origin', '*')
            self.send_header('Content-Type:', 'application/json; charset=UTF-8')
            self.end_headers()
            config.logging.debug("Finish writing the header and start writing JSON response.")
            json_encoded = json.dumps(response)
            config.logging.debug(json_encoded)
            self.wfile.write(json_encoded)
            config.logging.debug("JSON Response written successfully.")
          except Exception as e:
            config.logging.exception('Exception occurs when writing back the response.')
          return
    
       def _handle_request(self, args, response):
          try:
            response["sysTime"] = long(round(time.time() * 1000))
            if not "cmd" in args:
               response["message"] = "Error: No command provided."
            else:
               response["message"] = args['cmd']
          except Exception as e:
              response["message"] = "Error: Exception occurs (check logs)."
              config.logging.exception('Exception occurs when handling request.')
    
       def do_POST(self):
           self.send_response(200)
           self.send_header('Content-type',     'text/html')
           self.end_headers()
           self.wfile.write("Nothing here :( ")
    
    def main():
       while True:
        try:
            httpd = HTTPServer(('', config.listening_port), MyHandler)
            sa = httpd.socket.getsockname()
            msg = "Serving HTTP on " + sa[0] + " port " + str(sa[1]) + "..."
            print msg
            config.logging.info(msg)
            httpd.serve_forever()
        except KeyboardInterrupt:
            print '^C received, shutting down server'
            config.logging.info('^C received, shutting down server')
            httpd.socket.close()
            break
        except Exception as e:
            httpd.shutdown()
            httpd.socket.close()
            print "Error occurs. Retry in 5 seconds"
            config.logging.exception('Exception occurs. Retry in 5 seconds.')
            time.sleep(5)
    
    if __name__ == '__main__':
        main()
    

Sorry for missing the module config.

    
    
    import logging
    
    # global variables
    listening_port = 9001
    logging.basicConfig(filename='error.log', level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s', datefmt='%m/%d/%Y %I:%M:%S %p')
    

One url for running this is `http://localhost:9001/?cmd=some_command`. Just
paste that into any browser after the server is started.

**Edit:**

I guess this has something to do with the KVM or the way it is set up on my
company machine. As my server is running on a KVM which can only be accessed
through ssh, every time I exit ssh session, invoking the url command above (in
chrome), wait until chrome notifies not connected, get back to the ssh
session, invoking the url command again, the error happened.

I put the debug message between the write response in the do_GET() in
`self.send_response(200)`. When the exception happens, this is the trace I
got. I guess it has something to do with `sys.stderr.write` inside
log_message.

    
    
    11/23/2012 10:58:29 AM - INFO - Handle the request
    11/23/2012 10:58:29 AM - INFO - /?cmd=get_info&uid=pp
    11/23/2012 10:58:29 AM - INFO - Write the header back to the client.
    11/23/2012 10:58:29 AM - ERROR - Exception occurs when writing back the response.
    Traceback (most recent call last):
       File "service.py", line 57, in do_GET
          self.send_response(200)
       File "/usr/lib/python2.7/BaseHTTPServer.py", line 385, in send_response
          self.log_request(code)
       File "/usr/lib/python2.7/BaseHTTPServer.py", line 422, in log_request
          self.requestline, str(code), str(size))
       File "/usr/lib/python2.7/BaseHTTPServer.py", line 458, in log_message
          format%args))
       IOError: [Errno 5] Input/output error
    

**EDIT:**

The issue went away when I override the `log_message()` to do nothing. I know
this only hides the issue, but at least it works temporarily for me.


Answer: The sys.stderr writes message to the standard output, usually the shell
console where you started the program. So if the shell console gets closed,
the sys.stderr.writes will raise IO Error.

So the root cause of the issue is the sys.stderr.write in the log_message()
function tries to write a message to the ssh shell console(standard output)
where you originally started your python program, so if you keep the ssh
session alive, the issue won't happen. But if you exit the original ssh
session, the shell console gets closed, then the sys.stderr.write can't write
message to the original console, and then IO error happens.... The log_message
usually writes something like this:

    
    
    127.0.0.1 - - [01/Dec/2015 20:16:32] "GET / HTTP/1.1" 200 -
    

So the solution is to run your python script with redirecting the standard
output to /dev/null as follows:

    
    
    python your_script.py > /dev/null 2>&1 &
    

Holp it helps.

