title: How to debug memory leak in python with cherrypy and pytorch
body: <p>I have cherripy python app, which eats all my RAM eventually.<br>
  <strong>How do I debug the memory leak?</strong></p>
  
  <p>I have a <code>pytorch</code> model, and I wrapped it into <code>cherrypy</code> to provide simple rest-api for inference.</p>
  
  <pre class="lang-py prettyprint-override"><code>cherrypy.tree.mount(MyServer(), '/')
  cherrypy.config.update({
      'server.socket_port': 8080,
      'server.socket_host': "0.0.0.0",
      'server.thread_pool': 30,
      'log.access_file': "access1.log",
      'log.screen': True
  })
  try:
      cherrypy.engine.start()
      cherrypy.engine.block()
  except KeyboardInterrupt:
      cherrypy.engine.stop()
  </code></pre>
  
  <p>Where <code>Myserver is</code>:</p>
  
  <pre class="lang-py prettyprint-override"><code>class MyServer(object):
      @cherrypy.expose
      def index(self):
          return "healthy"
  
      @cherrypy.expose('/inference')
      def inference(self):
          cl = cherrypy.request.headers['Content-Length']
          rawbody = cherrypy.request.body.read(int(cl))
          body = json.loads(rawbody)
  
          result = doSomePyTorchStuff(body)
  
          return json.dumps(result)
  </code></pre>
  
  <p>Initially, the app takes around 800mb of ram, which mostly to pick up some binaries (pre-trained models) into the memory.
  With every new request, the memory usage keeps growing, and after 2 days saw the app took full <code>14G</code> of memory on my GPU doing nothing!</p>
  
  <p>I see the same result when running it on CPU instead of GPU. <code>23G</code> are consumed easily and even faster.<br>
  I do run my app in a docker container, but when running without it - the result is the same - memory is constantly consumed.  </p>
  
  <p>Can someone point me in the right direction?
  At this moment I can't narrow down the problem, not even sure if it's because of <code>cherrypy</code> or because of <code>pytorch</code>, or because something else.</p>