title: Apache Kafka: How to receive latest message from Kafka?
body: <p>I am consuming and processing messages in the Kafka consumer application using Spark in Scala. Sometimes it takes little more time than usual to process messages from Kafka message queue. At that time I need to consume latest message, ignoring the earlier ones which have been published by the producer and yet to be consumed.</p>
  
  <p>Here is my consumer code:</p>
  
  <pre><code>object KafkaSparkConsumer extends MessageProcessor {
  
  def main(args: scala.Array[String]): Unit = {
    val properties = readProperties()
  
    val streamConf = new SparkConf().setMaster("local[*]").setAppName("Kafka-Stream")
    val ssc = new StreamingContext(streamConf, Seconds(1))
  
    val group_id = Random.alphanumeric.take(4).mkString("dfhSfv")
    val kafkaParams = Map("metadata.broker.list"         -&gt;  properties.getProperty("broker_connection_str"), 
                        "zookeeper.connect"              -&gt;  properties.getProperty("zookeeper_connection_str"), 
                        "group.id"                       -&gt;  group_id, 
                        "auto.offset.reset"              -&gt;  properties.getProperty("offset_reset"),
                        "zookeeper.session.timeout"      -&gt;  properties.getProperty("zookeeper_timeout"))
  
    val msgStream = KafkaUtils.createStream[scala.Array[Byte], String, DefaultDecoder, StringDecoder](
                        ssc,
                        kafkaParams,
                        Map("moved_object" -&gt; 1),
                        StorageLevel.MEMORY_ONLY_SER
                        ).map(_._2)
  
    msgStream.foreachRDD { x =&gt;  
      x.foreach {  
        msg =&gt; println("Message: "+msg)
        processMessage(msg)
      }    
    }                         
    ssc.start()
    ssc.awaitTermination()
    }  
  }
  </code></pre>
  
  <p>Is there any way to make sure the consumer always gets the most recent message in the consumer application? Or do I need to set any property in Kafka configuration to achieve the same? </p>
  
  <p>Any help on this would be greatly appreciated. Thank you</p>