memory consumption and lifetime of temporaries
<p>I've a python code where the memory consumption steadily grows with time. While there are several objects which can legitimately grow quite large, I'm trying to understand whether the memory footprint I'm observing is due to these objects, or is it just me littering the memory with temporaries which don't get properly disposed of --- Being a recent convert from a world of manual memory management, I guess I just don't exactly understand some very basic aspects of how the python runtime deals with temporary objects. </p>

<p>Consider a code with roughly this general structure (am omitting irrelevant details): </p>

<pre><code>def tweak_list(lst):
    new_lst = copy.deepcopy(lst)
    if numpy.random.rand() &gt; 0.5:
        new_lst[0] += 1  # in real code, the operation is a little more sensible :-)
        return new_lst
    else:
        return lst


lst = [1, 2, 3]
cache = {}

# main loop
for step in xrange(some_large_number):

    lst = tweak_list(lst)    # &lt;&lt;-----(1)

    # do something with lst here, cut out for clarity

    cache[tuple(lst)] = 42   # &lt;&lt;-----(2)

    if step%chunk_size == 0:
        # dump the cache dict to a DB, free the memory (?)
        cache = {}           # &lt;&lt;-----(3)
</code></pre>

<p>Questions:</p>

<ol>
<li>What is the lifetime of a <code>new_list</code> created in a <code>tweak_list</code>? Will it be destroyed on exit, or will it be garbage collected (at which point?). Will repeated calls to <code>tweak_list</code> generate a gazillion of small lists lingering around for a long time? </li>
<li>Is there a temporary creation when converting a <code>list</code> to a <code>tuple</code> to be used as a <code>dict</code> key?</li>
<li>Will setting a <code>dict</code> to an empty one release the memory?</li>
<li>Or, am I approaching the issue at hand from a completely wrong perspective? </li>
</ol>
