Python Multiprocessing Kill Processes
Question: I am learning how to use the Python multiprocessing library. However, while I
am going through some of the examples, I ended up with many python processes
running in my background.

One of the [example](http://docs.python.org/2/library/multiprocessing.html)
looks like below:

    
    
    from multiprocessing import Process, Lock
    
    def f(l, i):
        l.acquire()
        print 'hello world', i
        l.release()
    
    if __name__ == '__main__':
        lock = Lock()
    
        for num in range(10):   # I changed the number of iterations from 10 to 1000...
            Process(target=f, args=(lock, num)).start()
    

Now here is a screen shot of my 'TOP' command:

    
    
    88950  Python       0.0  00:00.00 1    0    9     91    1584K  5856K  2320K  1720K  2383M  82441 1     sleeping 1755113321 799
    88949  Python       0.0  00:00.00 1    0    9     91    1584K  5856K  2320K  1720K  2383M  82441 1     sleeping 1755113321 798
    88948  Python       0.0  00:00.00 1    0    9     91    1580K  5856K  2316K  1716K  2383M  82441 1     sleeping 1755113321 797
    88947  Python       0.0  00:00.00 1    0    9     91    1580K  5856K  2316K  1716K  2383M  82441 1     sleeping 1755113321 796
    88946  Python       0.0  00:00.00 1    0    9     91    1576K  5856K  2312K  1712K  2383M  82441 1     sleeping 1755113321 795
    88945  Python       0.0  00:00.00 1    0    9     91    1576K  5856K  2312K  1712K  2383M  82441 1     sleeping 1755113321 794
    88944  Python       0.0  00:00.00 1    0    9     91    1576K  5856K  2312K  1712K  2383M  82441 1     sleeping 1755113321 794
    88943  Python       0.0  00:00.00 1    0    9     91    1572K  5856K  2308K  1708K  2383M  82441 1     sleeping 1755113321 792
    88942  Python       0.0  00:00.00 1    0    9     91    1568K  5856K  2304K  1708K  2383M  82441 1     sleeping 1755113321 790
    88941  Python       0.0  00:00.00 1    0    9     91    1564K  5856K  2300K  1704K  2383M  82441 1     sleeping 1755113321 789
    88938  Python       0.0  00:00.00 1    0    9     91    1564K  5856K  2300K  1704K  2383M  82441 1     sleeping 1755113321 788
    88936  Python       0.0  00:00.00 1    0    9     91    1576K  5856K  2296K  1716K  2383M  82441 1     sleeping 1755113321 787
    88935  Python       0.0  00:00.00 1    0    9     91    1560K  5856K  2296K  1700K  2383M  82441 1     sleeping 1755113321 787
    88934  Python       0.0  00:00.00 1    0    9     91    1560K  5856K  2296K  1700K  2383M  82441 1     sleeping 1755113321 786
    88933  Python       0.0  00:00.00 1    0    9     91    1556K  5856K  2292K  1696K  2383M  82441 1     sleeping 1755113321 785
    88932  Python       0.0  00:00.00 1    0    9     91    1556K  5856K  2292K  1696K  2383M  82441 1     sleeping 1755113321 784
    88931  Python       0.0  00:00.00 1    0    9     91    1552K  5856K  2288K  1692K  2383M  82441 1     sleeping 1755113321 783
    88930  Python       0.0  00:00.00 1    0    9     91    1612K  5856K  2288K  1752K  2383M  82441 1     sleeping 1755113321 783
    88929  Python       0.0  00:00.00 1    0    9     91    1588K  5856K  2288K  1728K  2383M  82441 1     sleeping 1755113321 782
    88927  Python       0.0  00:00.00 1    0    9     91    1608K  5856K  2284K  1748K  2383M  82441 1     sleeping 1755113321 781
    88926  Python       0.0  00:00.00 1    0    9     91    1548K  5856K  2284K  1688K  2383M  82441 1     sleeping 1755113321 780
    88924  Python       0.0  00:00.00 1    0    9     91    1556K  5856K  2276K  1700K  2383M  82441 1     sleeping 1755113321 778
    88923  Python       0.0  00:00.00 1    0    9     91    1540K  5856K  2276K  1684K  2383M  82441 1     sleeping 1755113321 777
    88922  Python       0.0  00:00.00 1    0    9     91    1540K  5856K  2276K  1684K  2383M  82441 1     sleeping 1755113321 776
    88921  Python       0.0  00:00.00 1    0    9     91    1536K  5856K  2272K  1680K  2383M  82441 1     sleeping 1755113321 774
    88920  Python       0.0  00:00.00 1    0    9     91    1528K  5856K  2264K  1672K  2383M  82441 1     sleeping 1755113321 771
    88919  Python       0.0  00:00.00 1    0    9     91    1528K  5856K  2264K  1672K  2383M  82441 1     sleeping 1755113321 771
    88918  Python       0.0  00:00.00 1    0    9     91    1528K  5856K  2264K  1672K  2383M  82441 1     sleeping 1755113321 770
    ....
    

  1. I don't know how to kill them in one go.

ps ... | grep python .... kill?

  2. what kind of python code do I need to add to avoid this miserable situation again. Thanks!


Answer: You need to .join() on your processes in a worker Queue, which will lock them
to the calling application until all of them succeed or kill when the parent
is killed, and run them in daemon mode.

[http://forums.xkcd.com/viewtopic.php?f=11&t=94726](http://forums.xkcd.com/viewtopic.php?f=11&t=94726)

[end daemon processes with multiprocessing
module](http://stackoverflow.com/questions/4792473/end-daemon-processes-with-
multiprocessing-module)

<http://docs.python.org/2/library/multiprocessing.html#the-process-class>

<http://www.python.org/dev/peps/pep-3143/#correct-daemon-behaviour>

