Why do I see random read errors with Python BaseHTTPServer?
Question: I have Python code that calls external HTTP services. I want to test this code
by setting up mock HTTP servers that imitate those external services. I do
this by starting a `BaseHTTPServer` in a separate thread, and then calling
that server from the main thread. It looks like this:

    
    
    import BaseHTTPServer, httplib, threading, time
    
    class MockHandler(BaseHTTPServer.BaseHTTPRequestHandler):
        def do_POST(self):
            self.send_response(200)
            self.send_header('Content-Type', 'application/json')
            self.end_headers()
            self.wfile.write('{"result": "success"}')
    
    class ServerThread(threading.Thread):
        def run(self):
            svr = BaseHTTPServer.HTTPServer(('127.0.0.1', 8540), MockHandler)
            svr.handle_request()
    
    ServerThread().start()
    time.sleep(0.1)             # Give the thread some time to get up
    
    conn = httplib.HTTPConnection('127.0.0.1', 8540)
    conn.request('POST', '/', 'foo=bar&baz=qux')
    resp_body = conn.getresponse().read()
    

However, some of the requests fail in the `read()` call, with `socket.error:
[Errno 104] Connection reset by peer`. I can reproduce it, with varying
frequency, on several machines with Python 2.6, though not with 2.7.

But the most interesting thing is, if I don’t send the POST data (i.e. if I
omit the third argument to `conn.request()`), the error does not occur.

What could this be?

Alternatively, is there another quick and easy way to set up mock HTTP servers
in Python?


Answer: "...in a separate thread, and then calling that server from the main thread."

Don't use threads for this kind of thing.

Use processes. `subprocess.Popen` (and your operating system's normal
features) will do a much, much better job of assuring that this works
properly.

