Multiprocessing pool 'apply_async' only seems to call function once
Question: I've been following the docs to try to understand multiprocessing pools. I
came up with this:

    
    
    import time
    from multiprocessing import Pool
    
    def f(a):
        print 'f(' + str(a) + ')'
        return True
    
    t = time.time()
    pool = Pool(processes=10)
    result = pool.apply_async(f, (1,))
    print result.get()
    pool.close()
    print ' [i] Time elapsed ' + str(time.time() - t)
    

I'm trying to use 10 processes to evaluate the function `f(a)`. I've put a
print statement in `f`.

This is the output I'm getting:

    
    
    $ python pooltest.py 
    f(1)
    True
     [i] Time elapsed 0.0270888805389
    

It appears to me that the function `f` is only getting evaluated once.

I'm likely not using the right method but the end result I'm looking for is to
run `f` with 10 processes simultaneously, and get the result returned by each
one of those process. So I would end with a list of 10 results (which may or
may not be identical).

The docs on multiprocessing are quite confusing and it's not trivial to figure
out which approach I should be taking and it seems to me that `f` should be
run 10 times in the example I provided above.


Answer: apply_async isn't meant to launch _multiple_ processes; it's just meant to
call the function with the arguments in one of the processes of the pool.
You'll need to make 10 calls if you want the function to be called 10 times.

First, note the docs on
[`apply()`](https://docs.python.org/3.4/library/multiprocessing.html#multiprocessing.pool.Pool.apply)
(emphasis added):

>
>     apply(func[, args[, kwds]])
>  
>
> Call func with arguments args and keyword arguments kwds. It blocks until
> the result is ready. Given this blocks, apply_async() is better suited for
> performing work in parallel. **Additionally, func is only executed in one of
> the workers of the pool.**

Now, in the docs for
[`apply_async()`](https://docs.python.org/3.4/library/multiprocessing.html#multiprocessing.pool.Pool.apply_async):

>
>     apply_async(func[, args[, kwds[, callback[, error_callback]]]])
>  
>
> A variant of the apply() method which returns a result object.

The difference between the two is just that apply_async returns immediately.
You can use `map()` to call a function multiple times, though if you're
calling with the same inputs, then it's a little redudant to create the list
of _the same_ argument just to have a sequence of the right length.

However, if you're calling different functions with the _same_ input, then
you're really just calling a higher order function, and you could do it with
`map` or
[`map_async()`](https://docs.python.org/3.4/library/multiprocessing.html#multiprocessing.pool.Pool.map_async)
like this:

    
    
    multiprocessing.map(lambda f: f(1), functions)
    

except that lambda functions aren't pickleable, so you'd need to use a defined
function (see [How to let Pool.map take a lambda
function](http://stackoverflow.com/q/4827432/1281433)). You can actually use
the builtin `apply()` (not the multiprocessing one) (although it's
deprecated):

    
    
    multiprocessing.map(apply,[(f,1) for f in functions])
    

It's easy enough to write your own, too:

    
    
    def apply_(f,*args,**kwargs):
      return f(*args,**kwargs)
    
    multiprocessing.map(apply_,[(f,1) for f in functions])
    

