title: Lightweight memory leak debugging on linux
body: <p>I looked for existing answers first and saw that <em>Valgrind</em> is everyone’s favorite tool for memory leak debugging on linux. Unfortunately <em>Valgrind</em> does not seem to work for my purposes. I will try to explain why.</p>
  
  <p>Constraints:</p>
  
  <ul>
  <li>The leak reproduces only in customer’s environment. Due to certain
  legal restrictions we have to work with existing binary. No rebuilds.</li>
  <li>In regular environment our application consumes ~10% CPU. Say, we can
  tolerate up to 10x CPU usage increase. <em>Valgrind</em> with default <em>memcheck</em>
  settings does much worse making our application unresponsive for long
  periods of time.</li>
  </ul>
  
  <p>What I need is an equivalent of Microsoft’s <em>UMDH</em>: turn on stack tracing for each heap allocation, then at certain point of time dump all allocations grouped by stacks and ordered by allocation count in descending order. Our app ships on both Windows and Linux platforms, so I know that performance on Windows under <em>UMDH</em> is still tolerable.</p>
  
  <p>Here are the tools/methods I considered</p>
  
  <ul>
  <li><em>Valgrind</em>'s <em>-memcheck</em> and <em>–massif</em> tools They do much more than needed (like scanning whole process memory for every allocation
  pointer), they are too slow, and they still don’t do exactly what I<br>
  need (dump callstacks sorted by counts), so I will have to write some
  scripts parsing the output</li>
  <li><em>dmalloc</em> library (dmalloc.com) requires new binary</li>
  <li>LeakTracer (<a href="http://www.andreasen.org/LeakTracer/" rel="noreferrer">http://www.andreasen.org/LeakTracer/</a>) Works only with C++
  <em>new/delete</em> (I need <em>malloc/free</em> as well), does not have group-by-stack
  and sort functionality</li>
  <li>Implementing the tool myself as .so library using LD_PRELOAD
  mechanism
  (<a href="https://stackoverflow.com/questions/6083337/overriding-malloc-using-the-ld-preload-mechanism">Overriding &#39;malloc&#39; using the LD_PRELOAD mechanism</a>)
  That will take at least a week given my coding-for-Linux skills and it feels
  like inventing a bicycle.</li>
  </ul>
  
  <p>Did I miss anything? Are there any lightweight <em>Valgrind</em> options or existing LD_PRELOAD tool?</p>