title: Amazon Kinesis & AWS Lambda Retries
body: <p>I'm very new to <strong>Amazon Kinesis</strong> so maybe this is just a problem in my understanding but in the <a href="https://aws.amazon.com/lambda/faqs/" rel="noreferrer">AWS Lambda FAQ</a> it says:</p>
  
  <blockquote>
    <p>The Amazon Kinesis and DynamoDB Streams records sent to your AWS Lambda function are strictly serialized, per shard. This means that if you put two records in the same shard, Lambda guarantees that your Lambda function will be successfully invoked with the first record before it is invoked with the second record. If the invocation for one record times out, is throttled, or encounters any other error, Lambda will retry until it succeeds (or the record reaches its 24-hour expiration) before moving on to the next record. The ordering of records across different shards is not guaranteed, and processing of each shard happens in parallel. </p>
  </blockquote>
  
  <p>My question is, what happens if for some reason some malformed data gets put onto a shard by a producer and when the Lambda function picks it up it errors out and then just keeps retrying constantly? This then means that the processing of that particular shard would be blocked for 24 hours by the error. </p>
  
  <p>Is the best practice to handle application errors like that by wrapping the problem in a custom error and sending this error downstream along with all the successfully processed records and let the consumer handle it? Of course, this still wouldn't help in the case of an unrecoverable error that crashed the program like a null pointer: again we'd be back to the blocking retry loop for the next 24 hours.</p>