title: Failed to find data source: Please deploy the application as per the deployment section of "Structured Streaming + Kafka Integration Guide"
body: <p>Hello I am trying to use pyspark + kafka in order to do this I execute this command in order to set up the kafka-cluster</p>
  <pre><code>zookeeper-server-start.sh $KAFKA_HOME/../config/zookeeper.properties
  
  kafka-server-start.sh $KAFKA_HOME/../config/*-0.properties &amp; kafka-server-start.sh $KAFKA_HOME/../config/*-1.properties
  </code></pre>
  <ul>
  <li>Spark version is - spark-3.2.0-bin-hadoop2-7</li>
  <li>Kafka version is - kafka_2.13-3.0.0</li>
  <li>pyspark version is 3.2.0</li>
  </ul>
  <p>The python code is:</p>
  <pre><code>spark_version = '3.2.0'
  os.environ['PYSPARK_SUBMIT_ARGS'] = '--packages org.apache.spark:spark-sql-kafka-0-10_2.12:{}'.format(spark_version)
  
  spark = SparkSession \
      .builder \
      .appName(&quot;TP3&quot;) \
      .getOrCreate()
  
  !spark-submit --class TP3 --packages org.apache.spark:spark-sql-kafka-0-10_2.12:3.2.0 TweetCount.ipynb
  </code></pre>
  <p>This return the following error:</p>
  <blockquote>
  <p>Error: Failed to load class TP3.</p>
  </blockquote>
  <p>And when I execute spark.readStream</p>
  <pre><code>consumer = KafkaConsumer('topic')
  df_kafka = spark.readStream \
      .format(&quot;kafka&quot;) \
      .option(&quot;kafka.bootstrap.servers&quot;, 'localhost:9092') \
      .option(&quot;subscribe&quot;, 'topic') \
      .load()
  </code></pre>
  <p>And I got this error:</p>
  <blockquote>
  <p>Failed to find data source: kafka. Please deploy the application as
  per the deployment section of &quot;Structured Streaming + Kafka
  Integration Guide&quot;.</p>
  </blockquote>
  <p>How can I execute the readstream in order to read from kafka with pyspark?</p>
  <p>Thanks</p>