Why only 1024 bytes are read in socketserver example
Question: I am reading through the documentation examples for python socketserver at
<https://docs.python.org/2/library/socketserver.html>

Why is the size specified as 1024 in the line `self.request.recv(1024)` inside
handle method. What happens if the data sent by the client is more than 1024
bytes ? Is it better to have a loop to read 1024 bytes until socket is empty ?
I have copied the example here :

    
    
    import SocketServer
    
    class MyTCPHandler(SocketServer.BaseRequestHandler):
        """
        The RequestHandler class for our server.
    
        It is instantiated once per connection to the server, and must
        override the handle() method to implement communication to the
        client.
        """
    
        def handle(self):
            # self.request is the TCP socket connected to the client
            self.data = self.request.recv(1024).strip() # why only 1024 bytes ?
            print "{} wrote:".format(self.client_address[0])
            print self.data
            # just send back the same data, but upper-cased
            self.request.sendall(self.data.upper())
    
    if __name__ == "__main__":
        HOST, PORT = "localhost", 9999
    
        # Create the server, binding to localhost on port 9999
        server = SocketServer.TCPServer((HOST, PORT), MyTCPHandler)
    
        # Activate the server; this will keep running until you
        # interrupt the program with Ctrl-C
        server.serve_forever()
    


Answer: When reading from a socket it's always required to make a loop.

The reason is that even if the source sent say 300 bytes over the network it's
possible for example that the data will arrive to the receiver as two separate
chunks of 200 bytes and 100 bytes.

For this reason when you specify a buffer size for `recv` you only say the
**maximum** amount you're willing to process, but the actual data amount
returned may be smaller.

There is no way to implement a "read until the end of the message" at the
Python level because the `send`/`recv` functions are simply wrappers of the
TCP socket interface and that is a **stream** interface, without message
boundaries (so there is no way to know if "all" the data has been received
from the source).

This also means that in many cases you will need to add your own boundaries if
you need to talk using messages (or you will need to use an higher-level
message-based network transport interface like [0MQ](http://zeromq.org/))

Note that "blocking mode" - when reading from a socket - only defines the
behavior when there is no data already received by the network layer of the
operating system: in that case, when blocking - the program will wait for a
chunk of data; if non-blocking instead - it will return immediately without
waiting. If there is any data already received by the computer, then the
`recv` call immediately returns even if the passed buffer size is bigger -
independently of the blocking/non-blocking setting.

Blocking mode doesn't mean that the `recv` call will wait for the buffer to be
filled.

**NOTE** : The Python documentation is indeed misleading on the behavior of
`recv` and hopefully will be fixed soon.

