title: Getting messages sent from python KafkaProducer
body: <p>My goal is to get data from non-file sources (i.e. generated within a program or sent though an API) and have it sent to a spark stream. To accomplish this, I'm sending the data through a <a href="https://github.com/dpkp/kafka-python" rel="nofollow noreferrer">python-based</a> <code>KafkaProducer</code>:</p>
  
  <pre><code>$ bin/zookeeper-server-start.sh config/zookeeper.properties &amp;
  $ bin/kafka-server-start.sh config/server.properties &amp;
  $ bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic my-topic
  $ python 
  Python 3.6.1| Anaconda custom (64-bit)
  &gt; from kafka import KafkaProducer
  &gt; import time
  &gt; producer = KafkaProducer(bootstrap_servers='localhost:9092', value_serializer=lambda v: json.dumps(v).encode('utf-8'))
  &gt; producer.send(topic = 'my-topic', value = 'MESSAGE ACKNOWLEDGED', timestamp_ms = time.time())
  &gt; producer.close()
  &gt; exit()
  </code></pre>
  
  <p>My issue is that nothing appears when checking the topic from the consumer shell script:</p>
  
  <pre><code>$ bin/kafka-console-consumer.sh --bootstrap-server localhost:2181 --topic my-topic
  ^C$
  </code></pre>
  
  <p>Is something missing or wrong here? I'm new to spark/kafka/messaging systems, so anything will help. The Kafka version is 0.11.0.0 (Scala 2.11) and no changes are made to the config files.</p>