title: python pool.map vs map
body: <p>I need to apply a complex function to a long series. Done sequentially this seems to be inefficient as I have access to a 12 core machine.</p>
  
  <p>Before making significant investment in this, i wrote up this simple version which compares pool.map and map. </p>
  
  <p>1) Surprisingly map is much much faster (see results below). </p>
  
  <p>2) And there is an overflow error in the pool function which does not come up on the map version.</p>
  
  <p>3) with a smaller array, the run time warning does not appear, but map is still faster.</p>
  
  <p>I am not a computer science guy (just a functional user) - any thoughts and suggestions? I went with pool.map because the async version may mess up the order of the series (which is a pain for me to sort out). </p>
  
  <p>SEE UPDATE BELOW : Based on Jdog's suggestion.</p>
  
  <p>Config: python v 2.7, 64 bit version, windows 7 </p>
  
  <pre><code>#-------------------------------------------------------------------------------
  # Name:        poolMap
  #-------------------------------------------------------------------------------
  import multiprocessing as mp
  import numpy as np
  import time
  
  def func(x):
     y=x*x
     return y
  
  def worker(inputs):
     num=mp.cpu_count()
     print 'num of cpus', num
     pool = mp.Pool(num)
     #inputs = list(inputs)
     #print "inputs type",type(inputs)
     results = pool.map(func, inputs)
     pool.close()
     pool.join()
     return results
  
  if __name__ == '__main__':
     series = np.arange(500000)
     start = time.clock()
     poolAnswer = worker(series)
     end = time.clock()
     print 'pool time' ,(end - start)
     start = time.clock()
     answer = map(func,series)
     end = time.clock()
     print 'map time', (end - start)
  </code></pre>
  
  <p>results:</p>
  
  <p>num of cpus 12</p>
  
  <p>pool time 2.40276007188</p>
  
  <p>D:\poolmap.py:19: RuntimeWarning: overflow encountered in long_scalars
        y=x*x</p>
  
  <p>map time 0.904187849745</p>
  
  ##############UPDATE
  
  <p>Using this func gave me the results I was looking for</p>
  
  <pre><code>def func(x):
      x=float(x)
      y=(((x*x)**0.35))*x+np.ma.sqrt((((x*x)**0.35)))
      return y 
  </code></pre>
  
  <p>results:
  num of cpus 12</p>
  
  <p>pool time 12.7410957475</p>
  
  <p>map time 45.4550067581</p>