title: Debugging "Managed memory leak detected" in Spark 1.6.0
body: <p>I've tried upgrading to Apache Spark 1.6.0 RC3. My application now spams these errors for nearly every task:</p>
  
  <pre><code>Managed memory leak detected; size = 15735058 bytes, TID = 830
  </code></pre>
  
  <p>I've set logging level for <code>org.apache.spark.memory.TaskMemoryManager</code> to <code>DEBUG</code> and see in the logs:</p>
  
  <pre><code>I2015-12-18 16:54:41,125 TaskSetManager: Starting task 0.0 in stage 7.0 (TID 6, localhost, partition 0,NODE_LOCAL, 3026 bytes)
  I2015-12-18 16:54:41,125 Executor: Running task 0.0 in stage 7.0 (TID 6)
  I2015-12-18 16:54:41,130 ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
  I2015-12-18 16:54:41,130 ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
  D2015-12-18 16:54:41,188 TaskMemoryManager: Task 6 acquire 5.0 MB for null
  I2015-12-18 16:54:41,199 ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
  I2015-12-18 16:54:41,199 ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
  D2015-12-18 16:54:41,262 TaskMemoryManager: Task 6 acquire 5.0 MB for null
  D2015-12-18 16:54:41,397 TaskMemoryManager: Task 6 release 5.0 MB from null
  E2015-12-18 16:54:41,398 Executor: Managed memory leak detected; size = 5245464 bytes, TID = 6
  </code></pre>
  
  <p>How do you debug these errors? Is there a way to log stack traces for allocations and deallocations, so I can find what leaks?</p>
  
  <p>I don't know much about the new unified memory manager (<a href="https://issues.apache.org/jira/browse/SPARK-10000">SPARK-10000</a>). Is the leak likely my fault or is it likely a Spark bug?</p>