How to properly detect closed socket in Python
Question: I have a python script that emits an `HTTP POST` with json body which I pipe
to `netcat`and all works fine:

    
    
    #!/usr/bin/python
    
    body = "{ \"key\": \"Test\", \"value\": \"Test\" }"
    
    print "POST /valSet HTTP/1.1\r"
    print "Host: 192.168.10.1\r"
    print "Content-Type: application/json\r"
    print "Accept: application/json\r"
    print "Content-Length: " + str(len(body)+2) + "\r"
    print "\r"
    
    print body
    
    $ ./valSet | nc 192.168.10.1 80
    HTTP/1.1 200 OK
    Server: Innovix
    Connection: close
    Transfer-Encoding: chunked
    Content-Type: application/json
    
    e
    {"success": 0}
    0
    $
    

As you can see above, the `HTTP` code is sent, `netcat` forwards it to port
80, and the response is successful. More importantly, the connection
immediately terminates after the last `0` appears in the chunked response.

However in my setting I can't rely on `netcat` being there, so I need a self-
contained program. I therefore rewrote the above as follows:

    
    
    def set():
        key = raw_input("Key: ")
        val = raw_input("Value: ")
        body = "{ \"key\": \"" + key + "\", \"value\": \"" + val + "\" }"
        send(body)
    
    def send(body):
        s.send("POST /valSet HTTP/1.1\r\n")
        s.send("Host: " + SERVER_IP + "\r\n")
        s.send("Content-Type: application/json\r\n")
        s.send("Accept: application/json\r\n")
        s.send("Content-Length: " + str(len(body)+2) + "\r\n")
        s.send("\r\n")
        s.send(body)
    
        # Get response
        while True:
            data = s.recv(1024) 
            print 'Response:', data
            if not data:
                break
    
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 
    s.connect((SERVER_IP, SERVER_PORT)) 
    set()
    

This again works, but with a severe drawback: `recv()` executes twice, then
hangs. On the server side it eventually times out with `Client timed out;
closing socket`.

**Question:** Why does the `netcat` version terminate immediately and my
python version blocks on `recv()` until the server hangs up?


Answer: netcat closes the connection because its input (e.g. your python script)
finished and thus stdin of netcat got closed. Your python script which tries
to send a HTTP request to the server hangs because you do make several
mistakes with using HTTP:

  * You set content-length to `len(body)+2`, but it it must be only the length of the body, e.g. `len(body)`. That's why the server is still waiting for data from your client.
  * You do an HTTP/1.1 request but don't deal with the implicit keep-alive you have but instead expect the server to close the connection. Also you don't deal with chunked responses.

I would recommend to use a http library for doing the hard work, unless you
are eager to learn all the details and pitfalls of doing HTTP. In this case I
would recommend you read the specification first, e.g. RFC2616 (or better:
RFC7230..RFC7235).

