Python secure websocket memory consumption
Question: I am writing a web socket server in python. I have tried the approach below
with txws, autobahn, and tornado, all with similar results.

I seem to have massive memory consumption with secure websockets and I cannot
figure out where or why this might be happening. Below is an example in
tornado, but I can provide examples in autobahn or txws.

    
    
    import tornado.httpserver
    import tornado.websocket
    import tornado.ioloop
    import tornado.web
    import json 
    
    class AuthHandler(tornado.websocket.WebSocketHandler):
    
        def open(self):
            print 'new connection for auth'
    
        def on_message(self, message):
            message = json.loads(message)
            client_id = message['client_id']
            if client_id not in app.clients:
                app.clients[client_id] = self
            self.write_message('Agent Recorded')
    
        def on_close(self):
            print 'auth connection closed'
    
    
    class MsgHandler(tornado.websocket.WebSocketHandler):
    
        def open(self):
            print 'new connection for msg'
    
        def on_message(self, message):
            message = json.loads(message)
            to_client = message['client_id']
            if to_client in app.clients:
                app.clients[to_client].write_message('You got a message')
    
        def on_close(self):
            print 'msg connection closed'
    
    
    app = tornado.web.Application([
        (r'/auth', AuthHandler),
        (r'/msg', MsgHandler)
    ])
    
    app.clients = {}
    
    
    if __name__ == "__main__":
        http_server = tornado.httpserver.HTTPServer(app, ssl_options={
            'certfile': 'tests/keys/server.crt',
            'keyfile': 'tests/keys/server.key'
        })
        http_server.listen(8000)
        tornado.ioloop.IOLoop.instance().start()
    

After making around 10,000 connections I find I am using around 700MB of
memory with SSL compared to 43MB without, and I never get it back unless I
kill the process. It seems like the problem is closely tied to the amount of
connections made rather than messages sent.

The consumption seems to happen independent of the client (I wrote my own
client and tried other clients).

Are secure websockets really that much more memory intensive that plain
websockets? Or is my server code not implementing it correctly?


Answer: I think the best solution is to use a real webserver (nginx apache) as a proxy
and let it manage the ssl layer.

