title: How to emulate multiprocessing.Pool.map() in AWS Lambda?
body: <p>Python on AWS Lambda does not support <code>multiprocessing.Pool.map()</code>, as documented in <a href="https://stackoverflow.com/questions/34005930/multiprocessing-semlock-is-not-implemented-when-running-on-aws-lambda">this other question</a>. Please note that the other question was asking why it doesn't work. This question is different, I'm asking how to emulate the functionality given the lack of underlying support.</p>
  
  <p>One of the answers to that other question gave us this code:</p>
  
  <pre><code># Python 3.6
  from multiprocessing import Pipe, Process
  
  def myWorkFunc(data, connection):
      result = None
  
      # Do some work and store it in result
  
      if result:
          connection.send([result])
      else:
          connection.send([None])
  
  
  def myPipedMultiProcessFunc():
  
      # Get number of available logical cores
      plimit = multiprocessing.cpu_count()
  
      # Setup management variables
      results = []
      parent_conns = []
      processes = []
      pcount = 0
      pactive = []
      i = 0
  
      for data in iterable:
          # Create the pipe for parent-child process communication
          parent_conn, child_conn = Pipe()
          # create the process, pass data to be operated on and connection
          process = Process(target=myWorkFunc, args=(data, child_conn,))
          parent_conns.append(parent_conn)
          process.start()
          pcount += 1
  
          if pcount == plimit: # There is not currently room for another process
              # Wait until there are results in the Pipes
              finishedConns = multiprocessing.connection.wait(parent_conns)
              # Collect the results and remove the connection as processing
              # the connection again will lead to errors
              for conn in finishedConns:
                  results.append(conn.recv()[0])
                  parent_conns.remove(conn)
                  # Decrement pcount so we can add a new process
                  pcount -= 1
  
      # Ensure all remaining active processes have their results collected
      for conn in parent_conns:
          results.append(conn.recv()[0])
          conn.close()
  
      # Process results as needed
  </code></pre>
  
  <p>Can this sample code be modified to support <code>multiprocessing.Pool.map()</code>?</p>
  
  <p><em>What have I tried so far</em></p>
  
  <p>I analysed the above code and I do not see a parameter for the function to be executed or the data, so I'm inferring that it does not perform the same function as <code>multiprocessing.Pool.map()</code>. It is not clear what the code does, other than demonstrating the building blocks that could be assembled into a solution.</p>
  
  <p><em>Is this a "write my code for me" question?</em></p>
  
  <p>Yes to some extent, it is. This issue impacts thousands of Python developers, and it would be far more efficient for the world economy, less green-house gas emissions, etc  if all of us share the same code, instead of forcing every SO user who encounters this to go and develop their own workaround. I hope I've done my part by distilling this into a clear question with the presumed building blocks ready to go.</p>