title: Memory leak with TensorFlow
body: <p>I have a memory leak with TensorFlow. I refered to <a href="https://stackoverflow.com/questions/35695183/tensorflow-memory-leak-even-while-closing-session">Tensorflow : Memory leak even while closing Session?</a> to address my issue, and I followed the advices of the answer, that seemed to have solved the problem. However it does not work here. </p>
  
  <p>In order to recreate the memory leak, I have created a simple example. First, I use this function (that I got here : <a href="https://stackoverflow.com/questions/276052/how-to-get-current-cpu-and-ram-usage-in-python">How to get current CPU and RAM usage in Python?</a>) to check the memory use of the python process : </p>
  
  <pre><code>def memory():
      import os
      import psutil
      pid = os.getpid()
      py = psutil.Process(pid)
      memoryUse = py.memory_info()[0]/2.**30  # memory use in GB...I think
      print('memory use:', memoryUse)
  </code></pre>
  
  <p>Then, everytime I call the <code>build_model</code> function, the use of memory increases.</p>
  
  <p>Here is the <code>build_model</code> function that has a memory leak : </p>
  
  <pre><code>def build_model():
  
      '''Model'''
  
      tf.reset_default_graph()
  
  
      with tf.Graph().as_default(), tf.Session() as sess:
          tf.contrib.keras.backend.set_session(sess)
  
          labels = tf.placeholder(tf.float32, shape=(None, 1))
          input = tf.placeholder(tf.float32, shape=(None, 1))
  
          x = tf.contrib.keras.layers.Dense(30, activation='relu', name='dense1')(input)
          x1 = tf.contrib.keras.layers.Dropout(0.5)(x)
          x2 = tf.contrib.keras.layers.Dense(30, activation='relu', name='dense2')(x1)
          y = tf.contrib.keras.layers.Dense(1, activation='sigmoid', name='dense3')(x2)
  
  
          loss = tf.reduce_mean(tf.contrib.keras.losses.binary_crossentropy(labels, y))
  
          train_step = tf.train.AdamOptimizer(0.004).minimize(loss)
  
          #Initialize all variables
          init_op = tf.global_variables_initializer()
          sess.run(init_op)
  
          sess.close()
  
      tf.reset_default_graph()
  
      return 
  </code></pre>
  
  <p>I would have thought that using the block <code>with tf.Graph().as_default(), tf.Session() as sess:</code> and then <strong>closing the session</strong> and <strong>calling <code>tf.reset_default_graph</code></strong> would clear all the memory used by TensorFlow. Apparently it does not.</p>
  
  <p>The memory leak can be recreated as following : </p>
  
  <pre><code>memory()
  build_model()
  memory()
  build_model()
  memory()
  </code></pre>
  
  <p>The output of this is (for my computer) :</p>
  
  <pre><code>memory use: 0.1794891357421875
  memory use: 0.184417724609375
  memory use: 0.18923568725585938
  </code></pre>
  
  <p>Clearly we can see that all the memory used by TensorFlow is not freed afterwards. Why?</p>
  
  <p>I plotted the use of memory over 100 iterations of calling <code>build_model</code>, and this is what I get : </p>
  
  <p><a href="https://i.stack.imgur.com/FzDlU.png" rel="noreferrer"><img src="https://i.stack.imgur.com/FzDlU.png" alt="Memory use over 100 iterations"></a></p>
  
  <p>I think that goes to show that there is a memory leak.</p>