title: How to rewind Kafka Offsets in spark structured streaming readstream
body: <p>I have a Spark Structured Streaming job which is configured to read data from Kafka. Please go through the code to check the <code>readStream()</code> with parameters to read the latest data from Kafka.</p>
  <p>I understand that <code>readStream()</code> reads from the first offset when a new query is started and not on resume.</p>
  <p>But I don't know how to start a new query every time I restart my job in IntelliJ.</p>
  <pre class="lang-scala prettyprint-override"><code>val kafkaStreamingDF = spark
    .readStream
    .format(&quot;kafka&quot;)
    .option(&quot;kafka.bootstrap.servers&quot;, AppProperties.getProp(AppConstants.PROPS_SERVICES_KAFKA_SERVERS))
    .option(&quot;subscribe&quot;, AppProperties.getProp(AppConstants.PROPS_SDV_KAFKA_TOPICS))
    .option(&quot;failOnDataLoss&quot;, &quot;false&quot;)
    .option(&quot;startingOffsets&quot;,&quot;earliest&quot;)
    .load()
    .selectExpr(&quot;CAST(value as STRING)&quot;, &quot;CAST(topic as STRING)&quot;)
  </code></pre>
  <p>I have also tried setting the offsets by <code>&quot;&quot;&quot;{&quot;topicA&quot;:{&quot;0&quot;:0,&quot;1&quot;:0}}&quot;&quot;&quot;</code></p>
  <p>Following is my writestream</p>
  <pre class="lang-scala prettyprint-override"><code>val query = kafkaStreamingDF
    .writeStream
    .format(&quot;console&quot;)
    .start()
  </code></pre>
  <p>Every time I restart my job in IntelliJ IDE, logs show that the offset has been set to latest instead of 0 or earliest.</p>
  <p>Is there way I can clean my checkpoint, in that case I don't know where the checkpoint directory is because in the above case I don't specify any checkpointing.</p>