title: Python: multiprocessing.map fails with queue.FULL
body: <p>I am using the <code>map()</code> function from </p>
  
  <pre><code> from concurrent.futures import ProcessPoolExecutor
  </code></pre>
  
  <p>in order to do a simple data parallelization.</p>
  
  <p>I want to process 400 files, using <code>map()</code> to call a processing function on them.</p>
  
  <pre><code>  infiles = glob.glob(os.path.join(input_path, '**/*.xls'), recursive=True) + glob.glob(os.path.join(input_path, '**/*.xlsx'), recursive=True) 
    outfiles = [os.path.join(os.path.dirname(infile), os.path.basename(infile).split('.')[0]+'.csv') for infile in infiles]
  
    with ProcessPoolExecutor(max_workers=None) as executor:
        executor.map(excel2csv, infiles, outfiles)
  </code></pre>
  
  <p>so <code>excel2csv()</code> should be called for each file, passing its desired input and output path. It will process each file independently, writing results to disc, and returns nothing.</p>
  
  <p>After about 100 files, the application throws an Exception, complaining about a full Queue.</p>
  
  <pre><code>Exception in thread Thread-1:
  Traceback (most recent call last):
    File "/home/mapa17/miniconda3/envs/pygng/lib/python3.5/threading.py", line 914, in _bootstrap_inner
      self.run()
    File "/home/mapa17/miniconda3/envs/pygng/lib/python3.5/threading.py", line 862, in run
      self._target(*self._args, **self._kwargs)
    File "/home/mapa17/miniconda3/envs/pygng/lib/python3.5/concurrent/futures/process.py", line 295, in _queue_management_worker
      shutdown_worker()
    File "/home/mapa17/miniconda3/envs/pygng/lib/python3.5/concurrent/futures/process.py", line 253, in shutdown_worker
      call_queue.put_nowait(None)
    File "/home/mapa17/miniconda3/envs/pygng/lib/python3.5/multiprocessing/queues.py", line 129, in put_nowait
      return self.put(obj, False)
    File "/home/mapa17/miniconda3/envs/pygng/lib/python3.5/multiprocessing/queues.py", line 83, in put
      raise Full
  queue.Full
  </code></pre>
  
  <p>The most similar problem I found is discussed <a href="https://stackoverflow.com/questions/31552716/multiprocessing-queue-full">here</a>.</p>
  
  <p>But in my case the data passed to the <em>worker</em> function is minimal (containing two strings). Checking the default queue size (from _multiprocessing.SemLock.SEM_VALUE_MAX) which is fare bigger than 400.</p>
  
  <p>Any ideas?
  Thank you</p>