title: Running lots of processes modifying the same list with Python Multiprocessing results in OSError: Too many open files
body: <p>I can't get the following code to execute successfully, always ending up with the following error:</p>
  <pre><code>OSError: [Errno 24] Too many open files
  with the most recent call on p.start()
  </code></pre>
  <p>As you can see I already tried to execute the processes in chunks of 500, as they run fine when only 500 are executed, however after the second loop I still receive the above error.
  I guess that the processes are not properly closed after execution, however I could not figure out how to check and properly close them...</p>
  <p>here is my code:</p>
  <pre><code>import multiprocessing
  
  manager = multiprocessing.Manager()
  manager_lists = manager.list()
  
  
  def multiprocessing_lists(a_list):
      thread_lists = []
      for r in range(400):
          thread_lists.append(a_list)
      manager_lists.extend(thread_lists)
  
  
  manager = multiprocessing.Manager()
  manager_lists = manager.list()
  processes = []
  counter = 0
  chunk_size = 500
  all_processes = 4000
  
  
  for i in range(0, all_processes, chunk_size):
  
      for j in range(chunk_size):
          a_list = [1,2,3,4,5,6,7,8,9]
          p = multiprocessing.Process(target=multiprocessing_lists, args=(a_list,))
          processes.append(p)
          p.start()
  
          counter += 1
      for process in processes:
          process.join()
          process.terminate()
  
  
  print(len(manager_lists))
  </code></pre>