title: Python multiprocessing Pool vs multiprocessing ThreadPool
body: <p>I have a list of image paths that I want to divide between processes OR threads so that each process processes some part of the list. Processing includes loading image from the disk, do some computation and return the result. I'm using Python 2.7 <code>multiprocessing.Pool</code></p>
  
  <p>Here's how I create worker processes</p>
  
  <pre><code>def ProcessParallel(classifier,path):
      files=glob.glob(path+"\*.png")
      files_sorted=sorted(files,key=lambda file_name:int(file_name.split('--')[1]))
      p = multiprocessing.Pool(processes=4,initializer=Initializer,initargs=(classifier,))
      data=p.map(LoadAndClassify, files_sorted)
      return data
  </code></pre>
  
  <p>The issue I'm facing it that when I log initialization time in my Intializer function, I came to know that Workers aren't initialized in parallel , rather each worker is initialized with a gap of 5 seconds , Here are the logs for reference</p>
  
  <pre><code>2016-08-08 12:38:32,043 - custom_logging - INFO - Worker started
  2016-08-08 12:38:37,647 - custom_logging - INFO - Worker started
  2016-08-08 12:38:43,187 - custom_logging - INFO - Worker started
  2016-08-08 12:38:48,634 - custom_logging - INFO - Worker started 
  </code></pre>
  
  <p>I've tried using <code>multiprocessing.pool.ThreadPool</code> instead which starts Workers at the same time.<br>
  I know how multiprocessing on Windows work and we have to place a <code>main guard</code> to protect our code from spawning infinite processes. The issue in my case is that I've hosted my script on IIS using FASTCGI and my script isn't main , It's being run by the FastCGI process (There's a wfastcgi.py script which is responsible for that). Now there is a main guard inside wfastcgi.py and the logs indicate that I'm not creating infinite no of processes.
  <br></p>
  
  <blockquote>
    <p>Now I want to know that what exactly is the reason behind
    multiprocessing Pool not creating worker threads simultaneously, I'll
    really appreciate any help.</p>
  </blockquote>
  
  <p>EDIT 1: Here's my Initializer function</p>
  
  <pre><code>def Initializer(classifier):
      global indexing_classifier
      logger.info('Worker started')
      indexing_classifier=classifier
  </code></pre>