Python: yield-and-delete
<p>How do I yield an object from a generator and forget it immediately, so that it doesn't take up memory?</p>

<p>For example, in the following function:</p>

<pre><code>def grouper(iterable, chunksize):
    """
    Return elements from the iterable in `chunksize`-ed lists. The last returned
    element may be smaller (if length of collection is not divisible by `chunksize`).

    &gt;&gt;&gt; print list(grouper(xrange(10), 3))
    [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]
    """
    i = iter(iterable)
    while True:
        chunk = list(itertools.islice(i, int(chunksize)))
        if not chunk:
            break
        yield chunk
</code></pre>

<p>I don't want the function to hold on to the reference to <code>chunk</code> after yielding it, as it is not used further and just consumes memory, even if all outside references are gone.</p>

<hr>

<p><strong>EDIT</strong>: using standard Python 2.5/2.6/2.7 from python.org.</p>

<hr>

<p><strong>Solution</strong> (proposed almost simultaneously by @phihag and @Owen): wrap the result in a (small) mutable object and return the chunk anonymously, leaving only the small container behind:</p>

<pre><code>def chunker(iterable, chunksize):
    """
    Return elements from the iterable in `chunksize`-ed lists. The last returned
    chunk may be smaller (if length of collection is not divisible by `chunksize`).

    &gt;&gt;&gt; print list(chunker(xrange(10), 3))
    [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]
    """
    i = iter(iterable)
    while True:
        wrapped_chunk = [list(itertools.islice(i, int(chunksize)))]
        if not wrapped_chunk[0]:
            break
        yield wrapped_chunk.pop()
</code></pre>

<p>With this memory optimization, you can now do something like:</p>

<pre><code> for big_chunk in chunker(some_generator, chunksize=10000):
     ... process big_chunk
     del big_chunk # big_chunk ready to be garbage-collected :-)
     ... do more stuff
</code></pre>
