<p>Does anybody know a "technique" to discover memory leaks caused by smart pointers? I am currently working on a large project written in <strong>C++</strong> that heavily uses smart pointers with reference counting. Obviously we have some memory leaks caused by smart pointers, that are still referenced somewhere in the code, so that their memory does not get free'd. It's very hard to find the line of code with the "needless" reference, that causes the corresponding object not to be free'd (although it's not of use any longer).</p>

<p>I found some advice in the web, that proposed to collect call stacks of the increment/decrement operations of the reference counter. This gives me a good hint, which piece of code has caused the reference counter to get increased or decreased.</p>

<p>But what I need is some kind of algorithm that groups the corresponding "increase/decrease call stacks" together. After removing these pairs of call stacks, I hopefully have (at least) one "increase call stack" left over, that shows me the piece of code with the "needless" reference, that caused the corresponding object not to be freed. Now it will be no big deal to fix the leak!</p>

<p>But has anybody an idea for an "algorithm" that does the grouping?</p>

<p>Development takes place under <strong>Windows XP</strong>.</p>

<p>(I hope someone understood, what I tried to explain ...)</p>

<p>EDIt: I am talking about leaks caused by circular references.</p>

[{'answer_id': 67251, 'author': 'Mike Stone', 'author_id': 122, 'author_profile': 'https://Stackoverflow.com/users/122', 'pm_score': 1, 'selected': False, 'text': '<p>If I were you I would take the log and write a quick script to do something like the following (mine is in Ruby):</p>\n\n<pre><code>def allocation?(line)\n  # determine if this line is a log line indicating allocation/deallocation\nend\n\ndef unique_stack(line)\n  # return a string that is equal for pairs of allocation/deallocation\nend\n\nallocations = []\nfile = File.new "the-log.log"\nfile.each_line { |line|\n  # custom function to determine if line is an alloc/dealloc\n  if allocation? line\n    # custom function to get unique stack trace where the return value\n    # is the same for a alloc and dealloc\n    allocations[allocations.length] = unique_stack line\n  end\n}\n\nallocations.sort!\n\n# go through and remove pairs of allocations that equal,\n# ideally 1 will be remaining....\nindex = 0\n\nwhile index &lt; allocations.size - 1\n  if allocations[index] == allocations[index + 1]\n    allocations.delete_at index\n  else\n    index = index + 1\n  end\nend\n\nallocations.each { |line|\n  puts line\n}\n</code></pre>\n\n<p>This basically goes through the log and captures each allocation/deallocation and stores a unique value for each pair, then sort it and remove pairs that match, see what\'s left.</p>\n\n<p>Update: Sorry for all the intermediary edits (I accidentally posted before I was done)</p>\n'}, {'answer_id': 67317, 'author': 'yrp', 'author_id': 7228, 'author_profile': 'https://Stackoverflow.com/users/7228', 'pm_score': 3, 'selected': False, 'text': "<p>The way I do it is simply:\n- on every AddRef() record call-stack,\n- matching Release() removes it.\nThis way at the end of the program I'm left with AddRefs() without maching Releases. No need to match pairs,</p>\n"}, {'answer_id': 67378, 'author': 'Community', 'author_id': -1, 'author_profile': 'https://Stackoverflow.com/users/-1', 'pm_score': 2, 'selected': False, 'text': "<p>What I have done to solve this is to override the <strong>malloc/new</strong> &amp; <strong>free/delete</strong> operators such that they keep track in a data structure as much as possible about the operation you are performing.</p>\n\n<p>For example, when overriding <strong>malloc/new</strong>, You can create a record of the caller's address, the amount of bytes requested, the assigned pointer value returned and a sequence ID so all your records can be sequenced  (I do not know if you deal with threads but you need to take that into account, too).</p>\n\n<p>When writing the <strong>free/delete</strong> routines, I also keep track of the caller's address and the pointer info. Then I look backwards into the list and try to match the <strong>malloc/new</strong> counterpart using the pointer as my key. If I don't find it, raise a red flag. </p>\n\n<p>If you can afford it, you can embed in your data the sequence ID to be absolutely sure who and when allocation call was made. The key here is to uniquely identify each transaction pair as much as we can.</p>\n\n<p>Then you will have a third routine displaying your memory allocations/deallocation  history, along with the functions invoking each transaction. (this can be accomplished by parsing the symbolic map out of your linker). You will know how much memory you will have allocated at any time and who did it.</p>\n\n<p>If you don't have enough resources to perform these transactions (my typical case for 8-bit microcontrollers), you can output the same information via a serial or TCP link to another machine with enough resources.</p>\n"}, {'answer_id': 67601, 'author': 'Doug T.', 'author_id': 8123, 'author_profile': 'https://Stackoverflow.com/users/8123', 'pm_score': 1, 'selected': False, 'text': '<p>For Windows, check out:</p>\n\n<p><a href="http://msdn.microsoft.com/en-us/library/c99kz476(VS.80).aspx" rel="nofollow noreferrer">MFC Memory Leak Detection</a></p>\n'}, {'answer_id': 68855, 'author': 'bk1e', 'author_id': 8090, 'author_profile': 'https://Stackoverflow.com/users/8090', 'pm_score': 2, 'selected': False, 'text': '<p>Since you said that you\'re using Windows, you may be able to take advantage of Microsoft\'s user-mode dump heap utility, <a href="http://msdn.microsoft.com/en-us/library/ff560206.aspx" rel="nofollow noreferrer">UMDH</a>, which comes with the <a href="http://www.microsoft.com/whdc/devtools/debugging/default.mspx" rel="nofollow noreferrer">Debugging Tools for Windows</a>. UMDH makes snapshots of your application\'s memory usage, recording the stack used for each allocation, and lets you compare multiple snapshots to see which calls to the allocator "leaked" memory. It also translates the stack traces to symbols for you using dbghelp.dll.</p>\n\n<p>There\'s also another Microsoft tool called "LeakDiag" that supports more memory allocators than UMDH, but it\'s a bit more difficult to find and doesn\'t seem to be actively maintained. The latest version is at least five years old, if I recall correctly.</p>\n'}, {'answer_id': 70055, 'author': 'yrp', 'author_id': 7228, 'author_profile': 'https://Stackoverflow.com/users/7228', 'pm_score': 2, 'selected': False, 'text': "<p>It's not a matter of finding a leak. In case of smart-pointers it'll most probably direct to some generic place like CreateObject(), which is being called thousands of time. It's a matter of determining what place in the code didnt call Release() on ref-counted object.</p>\n"}, {'answer_id': 70674, 'author': '0124816', 'author_id': 11521, 'author_profile': 'https://Stackoverflow.com/users/11521', 'pm_score': 1, 'selected': False, 'text': '<p>I am a big fan of <a href="http://goog-perftools.sourceforge.net/doc/heap_checker.html" rel="nofollow noreferrer">Google\'s Heapchecker</a> -- it will not catch all leaks, but it gets most of them. (<strong>Tip</strong>: Link it into all your unittests.)</p>\n'}, {'answer_id': 72902, 'author': 'Drealmer', 'author_id': 12291, 'author_profile': 'https://Stackoverflow.com/users/12291', 'pm_score': 2, 'selected': False, 'text': '<p>If you can reproduce the leak in a deterministic way, a simple technique I often used is to number all your smart pointers in their order of construction (use a static counter in the constructor), and report this ID together with the leak. Then run the program again, and trigger a DebugBreak() when the smart pointer with the same ID gets constructed.</p>\n\n<p>You should also consider this great tool : <a href="http://www.codeproject.com/KB/applications/visualleakdetector.aspx" rel="nofollow noreferrer" title="Visual Leak Detector">http://www.codeproject.com/KB/applications/visualleakdetector.aspx</a></p>\n'}, {'answer_id': 73119, 'author': 'Joe', 'author_id': 12567, 'author_profile': 'https://Stackoverflow.com/users/12567', 'pm_score': 2, 'selected': False, 'text': '<p>What I do is wrap the smart pointer with a class that takes <strong>FUNCTION</strong> and <strong>LINE</strong> parameters. Increment a count for that function and line every time the constructor is called, and decrement the count every time the destructor is called. then, write a function that dumps the function/line/count information. That tells you where all of your references were created</p>\n'}, {'answer_id': 78553, 'author': 'paercebal', 'author_id': 14089, 'author_profile': 'https://Stackoverflow.com/users/14089', 'pm_score': 5, 'selected': True, 'text': '<p>Note that one source of leaks with <strong>reference-counting smart pointers</strong> are pointers with <strong>circular dependancies</strong>. For example, A have a smart pointer to B, and B have a smart pointer to A. Neither A nor B will be destroyed. You will have to find, and then break the dependancies.</p>\n\n<p>If possible, use boost smart pointers, and use shared_ptr for pointers which are supposed to be owners of the data, and weak_ptr for pointers not supposed to call delete.</p>\n'}, {'answer_id': 189328, 'author': 'Constantin', 'author_id': 20310, 'author_profile': 'https://Stackoverflow.com/users/20310', 'pm_score': 2, 'selected': False, 'text': '<p>To detect reference cycles you need to have a graph of all reference-counted objects. Such a graph is not easy to construct, but it can be done.</p>\n\n<p>Create a global <code>set&lt;CRefCounted*&gt;</code> to register living reference-counted objects. This is easier if you have common AddRef() implementation - just add <code>this</code> pointer to the set when object\'s reference count goes from 0 to 1. Similarly, in Release() remove object from the set when it\'s reference count goes from 1 to 0.</p>\n\n<p>Next, provide some way to get the set of referenced objects from each <code>CRefCounted*</code>. It could be a <code>virtual set&lt;CRefCounted*&gt; CRefCounted::get_children()</code> or whatever suits you. Now you have a way to walk the graph.</p>\n\n<p>Finally, implement your favorite algorithm for <a href="http://coding.derkeiler.com/Archive/General/comp.theory/2004-02/0462.html" rel="nofollow noreferrer">cycle detection in a directed graph</a>. Start the program, create some cycles and run cycle detector. Enjoy! :)</p>\n'}, {'answer_id': 15546730, 'author': 'mariosoft', 'author_id': 2194940, 'author_profile': 'https://Stackoverflow.com/users/2194940', 'pm_score': 0, 'selected': False, 'text': '<p>First step could be to know what class is leaking.\nOnce you know it, you can find who is increasing the reference:\n1. put a breakpoint on the constructor of class that is wrapped by shared_ptr.\n2. step in with debugger inside shared_ptr when its increasing the reference count: look at variable pn->pi_->use_count_\nTake the address of that variable by evaluating expression (something like this: &amp;this->pn->pi_.use_count_), you will get an address\n3. In visual studio debugger, go to Debug->New Breakpoint->New Data Breakpoint...\nEnter the address of the variable \n4. Run the program. Your program will stop every time when some point in the code is increasing and decreasing the reference counter.\nThen you need to check if those are matching.</p>\n'}]
['https://Stackoverflow.com/questions/67174', 'https://Stackoverflow.com', 'https://Stackoverflow.com/users/2012356/']