title: How to save latest offset that Spark consumed to ZK or Kafka and can read back after restart
body: <p>I am using <code>Kafka 0.8.2</code> to receive data from AdExchange then I use <code>Spark Streaming 1.4.1</code> to store data to <code>MongoDB</code>.</p>
  
  <p>My problem is when I restart my <code>Spark Streaming</code> Job for instance like update new version, fix bug, add new features. It will continue read the latest <code>offset</code> of <code>kafka</code> at the time then I will lost data AdX push to kafka during restart the job.</p>
  
  <p>I try something like <code>auto.offset.reset -&gt; smallest</code> but it will receive from 0 -> last then data was huge and duplicate in db.</p>
  
  <p>I also try to set specific <code>group.id</code> and <code>consumer.id</code> to <code>Spark</code> but it the same.</p>
  
  <p>How to save the latest <code>offset</code> spark consumed to <code>zookeeper</code> or <code>kafka</code> then can read back from that to latest <code>offset</code>?</p>