Python: SocketServer closes TCP connection unexpectedly
Question: I would like to implement a TCP/IP network client application that sends
requests to a Python
[SocketServer](https://docs.python.org/2/library/socketserver.html) and
expects responses in return. I have started out with the official Python
[SocketServer sample
code](https://docs.python.org/2/library/socketserver.html#socketserver-
tcpserver-example):

`server.py:`

    
    
    #!/usr/bin/env python
    # encoding: utf-8
    
    import SocketServer
    
    class MyTCPHandler(SocketServer.StreamRequestHandler):
    
        def handle(self):
            request  = self.rfile.readline().strip()
            print "RX [%s]: %s" % (self.client_address[0], request)
    
            response = self.processRequest(request)
    
            print "TX [%s]: %s" % (self.client_address[0], response)
            self.wfile.write(response)
    
        def processRequest(self, message):
            if   message == 'request type 01':
                return 'response type 01'
            elif message == 'request type 02':
                return 'response type 02'
    
    if __name__ == "__main__":
        server = SocketServer.TCPServer(('localhost', 12345), MyTCPHandler)
        server.serve_forever()
    

`client.py:`

    
    
    #!/usr/bin/env python
    # encoding: utf-8
    
    import socket
    
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    
    try:
        sock.connect(('127.0.0.1', 12345))
    
        data = 'request type 01'
        sent = sock.sendall(data + '\n')
        if sent == 0:
            raise RuntimeError("socket connection broken")
    
        received = sock.recv(1024)
        print "Sent:     {}".format(data)
        print "Received: {}".format(received)
    
        data = 'request type 02'
        sent = sock.sendall(data + '\n')
        if sent == 0:
            raise RuntimeError("socket connection broken")
    
        received = sock.recv(1024)
        print "Sent:     {}".format(data)
        print "Received: {}".format(received)
    
    except Exception as e:
        print e
    
    finally:
        sock.close()
    

`server.py` output:

    
    
    RX [127.0.0.1]: request type 01
    TX [127.0.0.1]: response type 01
    

`client.py` output:

    
    
    Sent:     request type 01
    Received: response type 01
    [Errno 54] Connection reset by peer
    

What am doing wrong ? It seems the server is closing the connection. How can I
make it stay open ?

Note: This is a follow-up question to [C++/Qt: QTcpSocket won't write after
reading](http://stackoverflow.com/q/29549878)

**Update (after[abarnert's answer](http://stackoverflow.com/a/29572128)):**

What I take away from this is that `SocketServer.StreamRequestHandler` is not
the most recent design and while it allows me to connect over a network, it
doesn't really support me with all the TCP/IP-related aspects I need to take
care of to implement robust communication.

This has been addressed in Python 3 with
[asyncio](https://docs.python.org/3.4/library/asyncio.html), but as the
project lives in Python 2, that's not an option. I have therefore implemented
the server and client described above in
[Twisted](https://twistedmatrix.com/):

`server.py:`

    
    
    #!/usr/bin/env python
    # encoding: utf-8
    
    from twisted.internet.protocol import Factory
    from twisted.protocols.basic import LineReceiver
    from twisted.internet import reactor
    
    class SimpleProtocol(LineReceiver):
    
        def connectionMade(self):
            print 'connectionMade'
    
        # NOTE: lineReceived(...) doesn't seem to get called
    
        def dataReceived(self, data):
            print 'dataReceived'
            print 'RX: %s' % data
    
            if   data == 'request type 01':
                response = 'response type 01'
            elif data == 'request type 02':
                response = 'response type 02'
            else:
                response = 'unsupported request'
    
            print 'TX: %s' % response
            self.sendLine(response)
    
    class SimpleProtocolFactory(Factory):
    
        def buildProtocol(self, addr):
            return SimpleProtocol()
    
    reactor.listenTCP(12345, SimpleProtocolFactory(), interface='127.0.0.1')
    reactor.run()
    

`client.py:`

    
    
    #!/usr/bin/env python
    # encoding: utf-8
    
    from twisted.internet import reactor
    from twisted.internet.protocol import Protocol
    from twisted.internet.endpoints import TCP4ClientEndpoint, connectProtocol
    
    class SimpleClientProtocol(Protocol):
        def sendMessage(self, msg):
            print "[TX]: %s" % msg
            self.transport.write(msg)
    
    def gotProtocol(p):
        p.sendMessage('request type 01')
        reactor.callLater(1, p.sendMessage, 'request type 02')
        reactor.callLater(2, p.transport.loseConnection)
    
    point = TCP4ClientEndpoint(reactor, '127.0.0.1', 12345)
    d = connectProtocol(point, SimpleClientProtocol())
    d.addCallback(gotProtocol)
    reactor.run()
    

The client doesn't close, but idles until `CTRL`+`C`. Twisted might take a
while to get my head around, but for the job at hand, it clearly seems more
reasonable to employ a tested and tried framework than to do all this
groundwork myself.

**NOTE:** This is continued at [Twisted XmlStream: How to connect to
events?](http://stackoverflow.com/q/29683769)


Answer: The problem here is that in a `TCPHandler`, a "request" is actually a complete
connection, from beginning to end.* Your handler gets called on `accept`, and
when you return from it, the socket gets closed.

If you want to build a request-response-protocol handler on top of that, which
processes multiple protocol-level requests on a single socket-level request,
you have to do that yourself (or use a higher-level framework). (Subclasses
like
[`BaseHTTPServer`](https://docs.python.org/2.7/library/basehttpserver.html)
demonstrate how to do this.)

For example, you can just use a loop within your `handle` function. Of course
you probably want to add an exception handler here and/or deal with EOF from
`rfile` (note `self.rfile.readline()` will return `''` for EOF, but `'\n'` for
a blank line, so you have to check it before calling `strip` unless you want a
blank line to mean "quit" in your protocol). For example:

    
    
    def handle(self):
        try:
            while True:
                request  = self.rfile.readline()
                if not request:
                    break
                request = request.rstrip()
                print "RX [%s]: %s" % (self.client_address[0], request)
    
                response = self.processRequest(request)
    
                print "TX [%s]: %s" % (self.client_address[0], response)
                self.wfile.write(response + '\n')
        except Exception as e:
            print "ER [%s]: %r" % (self.client_address[0], e)
        print "DC [%s]: disconnected" % (self.client_address[0])
    

* * *

This will _often_ work with your existing client, at least over localhost on
an unloaded machine, but it's not actually correct, and "often works" is
rarely good enough. See [TCP sockets are byte streams, not message
streams](http://stupidpythonideas.blogspot.com/2013/05/sockets-are-byte-
streams-not-message.html) for a longer discussion, but briefly, you _also_
need to do the stuff mentioned in [David Schwarz's
answer](http://stackoverflow.com/a/29571816/908494): append newlines to what
you write from the server (which I already did above), and have the client
read line by line instead of just trying to read 1024 bytes at a time (which
you can do by writing your own buffer-and-split-lines code, or just by using
the
[`makefile`](https://docs.python.org/2.7/library/socket.html#socket.socket.makefile)
method, so it can use `rfile.readline()` just like the server side does.)

Not fixing the client won't cause the problems that answer claims, but it
_will_ cause problems like this:

    
    
    Sent:     request type 01
    Received: resp
    Sent:     request type 02
    Received: onse type 01
    response typ
    

And you can see that in a real program that actually tried to process the
responses programmatically, a response like `resp` or `onse type 01\nresponse
typ` isn't going to be very useful…

* * *

* Note that `SocketServer` is an ancient design that nobody really loves. There's a reason Python 3 added [`asyncio`](https://docs.python.org/3.4/library/asyncio.html), and that people in Python 2 usually use third-party frameworks like [Twisted](https://twistedmatrix.com/) and [gevent](http://www.gevent.org/). They're both simpler for simple tasks, and more flexible/powerful (and a lot more efficient) for complex tasks.

