title: Tracking Async Lambda execution on AWS
body: <p>I am trying to build a process that invokes AWS lambda, which then utilizes AWS SNS to send messages that trigger more lambdas. Each such triggered lambdas write an output file to S3. The process is as depicted below -</p>
  
  <p><a href="https://i.stack.imgur.com/U30KO.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/U30KO.png" alt="enter image description here"></a> </p>
  
  <p>My question is this - How can I know that all lambdas are done with writing files? I want to execute another process that collects all these files and does merging. I could think of two obvious ways - </p>
  
  <ol>
  <li>Constantly monitor s3 for as many output files as SNS messages. Once, total count reaches, invoke the final merging lambda.</li>
  <li>Use a db as sync source,  write counts for that particular job/session and keep monitoring it till the count reaches SNS messages count.</li>
  </ol>
  
  <p>Both solutions require constant polling, which i would like to avoid. I want to do this in an event driven manner. I was hoping for Amazon SQS would come to my rescue with some sort of "empty queue lambda trigger", but SQS only supports lambdas triggering on new messages. Is there any known way to achieve this in an event driven manner in AWS? Your suggestions/comments/answers are much appreciated. </p>