title: Python: Writing to a single file with queue while using multiprocessing Pool
body: <p>I have hundreds of thousands of text files that I want to parse in various ways.  I want to save the output to a single file without synchronization problems.  I have been using multiprocessing pool to do this to save time, but I can't figure out how to combine Pool and Queue.  </p>
  
  <p>The following code will save the infile name as well as the maximum number of consecutive "x"s in the file.  However, I want all processes to save results to the same file, and not to different files as in my example.  Any help on this would be greatly appreciated.</p>
  
  <pre><code>import multiprocessing
  
  with open('infilenamess.txt') as f:
      filenames = f.read().splitlines()
  
  def mp_worker(filename):
   with open(filename, 'r') as f:
        text=f.read()
        m=re.findall("x+", text)
        count=len(max(m, key=len))
        outfile=open(filename+'_results.txt', 'a')
        outfile.write(str(filename)+'|'+str(count)+'\n')
        outfile.close()
  
  def mp_handler():
      p = multiprocessing.Pool(32)
      p.map(mp_worker, filenames)
  
  if __name__ == '__main__':
      mp_handler()
  </code></pre>