title: Python socket stress concurrency
body: <p>I need a Python TCP server that can handle at least tens of thousands  of concurrent socket connections. I was trying to test Python SocketServer package capabilities in both multiprocessor and multithreaded modes, but both were far from desired performance.</p>
  
  <p>At first, I'll describe client, because it's common for both cases.</p>
  
  <p><strong>client.py</strong></p>
  
  <pre><code>import socket
  import sys
  import threading
  import time
  
  
  SOCKET_AMOUNT = 10000
  HOST, PORT = "localhost", 9999
  data = " ".join(sys.argv[1:])
  
  
  def client(ip, port, message):
      sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
      sock.connect((ip, port))
      while 1:
          sock.sendall(message)
          time.sleep(1)
      sock.close()
  
  
  for i in range(SOCKET_AMOUNT):
      msg = "test message"
      client_thread = threading.Thread(target=client, args=(HOST, PORT, msg))
      client_thread.start()
  </code></pre>
  
  <p>Multiprocessor server:</p>
  
  <p><strong>foked_server.py</strong></p>
  
  <pre><code>import os
  import SocketServer
  
  
  class ForkedTCPRequestHandler(SocketServer.BaseRequestHandler):
  
      def handle(self):
          cur_process = os.getpid()
          print "launching a new socket handler, pid = {}".format(cur_process)
          while 1:
              self.request.recv(4096)
  
  
  class ForkedTCPServer(SocketServer.ForkingMixIn, SocketServer.TCPServer):
      pass
  
  
  if __name__ == "__main__":
      HOST, PORT = "localhost", 9999
  
      server = ForkedTCPServer((HOST, PORT), ForkedTCPRequestHandler)
      print "Starting Forked Server"
      server.serve_forever()
  </code></pre>
  
  <p>Multithreaded server:</p>
  
  <p><strong>threaded_server.py</strong></p>
  
  <pre><code>import threading
  import SocketServer
  
  
  class ThreadedTCPRequestHandler(SocketServer.BaseRequestHandler):
  
      def handle(self):
          cur_thread = threading.current_thread()
          print "launching a new socket handler, thread = {}".format(cur_thread)
          while 1:
              self.request.recv(4096)
  
  
  class ThreadedTCPServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
      pass
  
  
  if __name__ == "__main__":
      HOST, PORT = "localhost", 9999
  
      server = ThreadedTCPServer((HOST, PORT), ForkedTCPRequestHandler)
      print "Starting Threaded Server"
      server.serve_forever()
  </code></pre>
  
  <p>In the first case, with <strong>forked_server.py</strong>, only 40 processes are created and approximately 20 of those start breaking in a while with the following error:</p>
  
  <blockquote>
    <p>error: [Errno 104] Connection reset by peer</p>
  </blockquote>
  
  <p>on a client side.</p>
  
  <p>Threaded version is much more durable and holds more than 4000 connections, but eventually starts showing</p>
  
  <blockquote>
    <p>gaierror: [Errno -5] No address associated with hostname</p>
  </blockquote>
  
  <p>The tests were made on my local machine, Kubuntu 14.04 x64 on kernel v3.13.0-32. These are the steps I've made to increase general performance of the system:</p>
  
  <ol>
  <li>Raise kernel limit on file handles: <code>sysctl -w fs.file-max=10000000</code></li>
  <li>Increase the connection backlog, <code>sysctl -w net.core.netdev_max_backlog = 2500</code></li>
  <li>Raise the maximum connections, <code>sysctl -w net.core.somaxconn = 250000</code></li>
  </ol>
  
  <p>So, the questions are:</p>
  
  <ol>
  <li>Were the tests correct, can I rely on those results? I'm new to all this Network/Socket   stuff, so please correct me in my conclusions.</li>
  <li>Is it really the multiprocessor/multithreaded approach not viable in a heavy loaded systems?</li>
  <li>If yes, what options do we have left? Asynchronous approach? Tornado/Twisted/Gevent frameworks?</li>
  </ol>