title: Reliable SQS Receiver for Spark Streaming
body: <p>I would like to have a Spark Streaming <strong>SQS Receiver</strong> which deletes SQS messages only <strong>after</strong> they were successfully stored on S3.</p>
  
  <p>For this a <strong>Custom Receiver</strong> can be implemented with the semantics of the <strong>Reliable Receiver</strong>.</p>
  
  <p>The <code>store(multiple-records)</code> call <a href="https://spark.apache.org/docs/latest/streaming-custom-receivers.html#receiver-reliability" rel="nofollow">blocks until</a> the given records have been stored and replicated inside Spark.</p>
  
  <p>If the <strong>write-ahead logs</strong> are enabled, all the data received from a receiver gets written <a href="https://spark.apache.org/docs/latest/streaming-programming-guide.html#checkpointing" rel="nofollow">into a write ahead log in the configuration checkpoint directory</a>. The checkpoint directory can be pointed to S3.</p>
  
  <p>After the <code>store(multiple-records)</code> blocking call finishes, are the records already stored in the checkpoint directory (and thus can be safely deleted from SQS)?</p>