title: Python Kafka consumer with offset management
body: <p>I am a newbie to Kafka and I am trying to set up a consumer in Kafka such that it reads messages published by Kafka Producer. 
  Correct me if I am wrong, the way I understood if Kafka consumer stores offset in ZooKeeper? However, I dont have a zookeeper instance running and want to poll lets say every 5 mins to see if there are any new messages published.</p>
  
  <p>So far, the code that I have is:</p>
  
  <pre><code>import logging
  from django.conf import settings
  import kafka
  import sys
  import json
  
  bootstrap_servers = ['localhost:8080']
  topicName = 'test-info'
  consumer = kafka.KafkaConsumer (topicName, group_id = 'test',bootstrap_servers = 
  bootstrap_servers,
  auto_offset_reset = 'earliest')
  
  count = 0
  #print(consumer.topic)
  try:
      for message in consumer:
          #print(type(message.value))
          print("\n")
          print("&lt;&gt;"*20)
          print ("%s:%d:%d: key=%s value=%s" % (message.topic, message.partition,message.offset, message.key, message.value))
          print("--"*20)
          info = json.loads(message.value)
  
          if info['event'] == "new_record" and info['data']['userId'] == "user1" and info['data']['details']['userTeam'] == "foo":
             count = count + 1
             print(count, info['data']['details']['team'], info['data']['details']['leadername'],info['data']['details']['category'])
          else:
              print("Skipping")
  
      print(count)
  
  
  except KeyboardInterrupt:
      sys.exit()
  </code></pre>
  
  <p>How can I save the offset such that next time it polls it reads incremental data? Any pointers will help.</p>