Does python's multiprocessing leak memory?
Question: I've narrowed down a piece of code to the following minimal (working?)
example:

    
    
    import multiprocessing
    
    def f(x): return x**2
    
    for n in xrange(2000):
        P = multiprocessing.Pool()
        sol = list(P.imap(f, range(20)))
    

When I run this on my computer (Ubuntu 12.04, 8-core), python proceeds to
consume all available memory and eventually it hangs the system due to an
unresponsive swap. I've heard that Python doesn't release memory until it's
finished, but I think it's a bit silly that this program should consume over
8Gb of RAM. If I want to create `multiprocessing.Pool` objects over and over,
how can I do so without reallocating new memory each time?


Answer: You're creating a new multiprocessing pool every loop iteration - don't you
only want one pool to which to feed your processing jobs?

    
    
    import multiprocessing
    
    def f(x): return x**2
    
    P = multiprocessing.Pool()
    for n in xrange(2000):
        sol = list(P.imap(f, range(20)))
    

