title: Get the sequence no of the message processing in Spark Streaming
body: <p>I am using Spark Structured streaming for processing the messages and I am using Java8. I am reading the message from the kafka and writing the message to the file and save the file in HDFS. </p>
  
  <p>I got a requirement like I need to write a sequence number along with the message to file. </p>
  
  <p>For example, if I get the first message from <code>kafka</code>, the output file content will be "message, 1" , for second message its "message,2" etc.. kind of count.</p>
  
  <p>if the message count reaches some threshold let say "message, 999999", then I need to reset the sequence from 1 again from the next message I received.</p>
  
  <p>if the spark streaming job is restarted, it should continue with the sequence where it left.  So I need to save this number in <code>HDFS</code> kind of <code>checkPointLocation</code>.</p>
  
  <p>What is the best approach I can use to implement this sequence. Can I use Accumulator to do that? or is there any other better to approach to implement during the distributed processing ?  or is it not possible in distributed processing?</p>