title: python multiprocessing,pathos slow
body: <p>Today I ran some code and i wanted to run it on my multicore cpu so where even I wrote map I changed it to pool.map.
  surprisingly my code ran slower even though it was using so much processing power or memory(to my knowledge).
  so i wrote this test, it uses pathos and multiprocessing.</p>
  
  <pre><code>from pathos.pools import ProcessPool
  from pathos.pools import ThreadPool
  #from pathos.pools import ParallelPool
  from pathos.pools import SerialPool
  from multiprocessing import Pool
  
  import time
  
  def timeit(method):
      def timed(*args, **kw):
          ts = time.time()
          result = method(*args, **kw)
          te = time.time()
          print ('%r (%r, %r) %2.2f sec' % \
                (method.__name__, args, kw, te-ts))
          return result
  
      return timed
  
  def times2(x):
      return 2*x
  
  @timeit
  def test(max,p):
      (p.map(times2, range(max)))
  
  def main():
      ppool = ProcessPool(4)
      tpool = ThreadPool(4)
      #parapool = ParallelPool(4)
      spool = SerialPool(4)
      pool = Pool(4)
      for i in range(8):
          max = 10**i
          print(max)
          print('ThreadPool')
          test(max,tpool)
          #print('ParallelPool')
          #test(max,parapool)
          print('SerialPool')
          test(max,spool)
          print('Pool')
          test(max,pool)
          print('ProcessPool')
          test(max,ppool)
          print('===============')
  
  
  if __name__ == '__main__':
      main()
  </code></pre>
  
  <p>these are the results</p>
  
  <pre><code>1
  ThreadPool
  'test' ((1, &lt;pool ThreadPool(nthreads=4)&gt;), {}) 0.00 sec
  SerialPool
  'test' ((1, &lt;pool SerialPool()&gt;), {}) 0.00 sec
  Pool
  'test' ((1, &lt;multiprocessing.pool.Pool object at 0x0000011E63D276A0&gt;), {}) 0.17 sec
  ProcessPool
  'test' ((1, &lt;pool ProcessPool(ncpus=4)&gt;), {}) 0.00 sec
  ===============
  10
  ThreadPool
  'test' ((10, &lt;pool ThreadPool(nthreads=4)&gt;), {}) 0.00 sec
  SerialPool
  'test' ((10, &lt;pool SerialPool()&gt;), {}) 0.00 sec
  Pool
  'test' ((10, &lt;multiprocessing.pool.Pool object at 0x0000011E63D276A0&gt;), {}) 0.00 sec
  ProcessPool
  'test' ((10, &lt;pool ProcessPool(ncpus=4)&gt;), {}) 0.01 sec
  ===============
  100
  ThreadPool
  'test' ((100, &lt;pool ThreadPool(nthreads=4)&gt;), {}) 0.00 sec
  SerialPool
  'test' ((100, &lt;pool SerialPool()&gt;), {}) 0.00 sec
  Pool
  'test' ((100, &lt;multiprocessing.pool.Pool object at 0x0000011E63D276A0&gt;), {}) 0.00 sec
  ProcessPool
  'test' ((100, &lt;pool ProcessPool(ncpus=4)&gt;), {}) 0.01 sec
  ===============
  1000
  ThreadPool
  'test' ((1000, &lt;pool ThreadPool(nthreads=4)&gt;), {}) 0.00 sec
  SerialPool
  'test' ((1000, &lt;pool SerialPool()&gt;), {}) 0.00 sec
  Pool
  'test' ((1000, &lt;multiprocessing.pool.Pool object at 0x0000011E63D276A0&gt;), {}) 0.00 sec
  ProcessPool
  'test' ((1000, &lt;pool ProcessPool(ncpus=4)&gt;), {}) 0.02 sec
  ===============
  10000
  ThreadPool
  'test' ((10000, &lt;pool ThreadPool(nthreads=4)&gt;), {}) 0.00 sec
  SerialPool
  'test' ((10000, &lt;pool SerialPool()&gt;), {}) 0.00 sec
  Pool
  'test' ((10000, &lt;multiprocessing.pool.Pool object at 0x0000011E63D276A0&gt;), {}) 0.00 sec
  ProcessPool
  'test' ((10000, &lt;pool ProcessPool(ncpus=4)&gt;), {}) 0.09 sec
  ===============
  100000
  ThreadPool
  'test' ((100000, &lt;pool ThreadPool(nthreads=4)&gt;), {}) 0.04 sec
  SerialPool
  'test' ((100000, &lt;pool SerialPool()&gt;), {}) 0.00 sec
  Pool
  'test' ((100000, &lt;multiprocessing.pool.Pool object at 0x0000011E63D276A0&gt;), {}) 0.01 sec
  ProcessPool
  'test' ((100000, &lt;pool ProcessPool(ncpus=4)&gt;), {}) 0.74 sec
  ===============
  1000000
  ThreadPool
  'test' ((1000000, &lt;pool ThreadPool(nthreads=4)&gt;), {}) 0.42 sec
  SerialPool
  'test' ((1000000, &lt;pool SerialPool()&gt;), {}) 0.00 sec
  Pool
  'test' ((1000000, &lt;multiprocessing.pool.Pool object at 0x0000011E63D276A0&gt;), {}) 0.17 sec
  ProcessPool
  'test' ((1000000, &lt;pool ProcessPool(ncpus=4)&gt;), {}) 7.54 sec
  ===============
  10000000
  ThreadPool
  'test' ((10000000, &lt;pool ThreadPool(nthreads=4)&gt;), {}) 4.57 sec
  SerialPool
  'test' ((10000000, &lt;pool SerialPool()&gt;), {}) 0.00 sec
  Pool
  'test' ((10000000, &lt;multiprocessing.pool.Pool object at 0x0000011E63D276A0&gt;), {}) 2.25 sec
  ProcessPool
  'test' ((10000000, &lt;pool ProcessPool(ncpus=4)&gt;), {}) 81.51 sec
  ===============
  </code></pre>
  
  <p>as you can see multiprocessing often beats ProcessPool and is even slower than SerialPool.
  I am running i5-2500 and I installed pathos today via pip</p>
  
  <pre><code>&gt;pip freeze
  colorama==0.3.9
  decorator==4.1.2
  dill==0.2.7.1
  helper-htmlparse==0.1
  htmldom==2.0
  lxml==4.0.0
  multiprocess==0.70.5
  pathos==0.2.1
  pox==0.2.3
  ppft==1.6.4.7.1
  py==1.4.34
  pyfs==0.0.8
  pyreadline==2.1
  pytest==3.2.2
  six==1.11.0
  </code></pre>
  
  <p>why does this happen?</p>