title: spark streaming + kafka - spark session API
body: <p>Appreciate your help to run a spark streaming program using spark 2.0.2. </p>
  
  <p>The run errors with <code>"java.lang.ClassNotFoundException: Failed to find data source: kafka"</code>. Modified POM file as below.</p>
  
  <p>Spark is being created but errors when the load from kafka is called. </p>
  
  <p><strong>Created spark session:</strong></p>
  
  <pre><code> val spark = SparkSession
              .builder()
              .master(master)
              .appName("Apache Log Analyzer Streaming from Kafka")
              .config("hive.metastore.warehouse.dir", hiveWarehouse)
              .config("fs.defaultFS", hdfs_FS)
              .enableHiveSupport()
              .getOrCreate()
  </code></pre>
  
  <p><strong>Creating kafka streaming:</strong></p>
  
  <pre><code>    val logLinesDStream = spark
        .readStream
        .format("kafka")
        .option("kafka.bootstrap.servers", "localhost:2181")
        .option("subscribe", topics)
        .load()
  </code></pre>
  
  <p><strong>Error message:</strong></p>
  
  <pre><code>Exception in thread "main" java.lang.ClassNotFoundException: Failed to find data source: kafka. Please find packages at http://spark-packages.org
  </code></pre>
  
  <p><strong>POM.XML:</strong></p>
  
  <pre><code>    &lt;scala.version&gt;2.10.4&lt;/scala.version&gt;
          &lt;scala.compat.version&gt;2.10&lt;/scala.compat.version&gt;
          &lt;spark.version&gt;2.0.2&lt;/spark.version&gt;
      &lt;dependencies&gt;
          &lt;dependency&gt;
              &lt;groupId&gt;org.scala-lang&lt;/groupId&gt;
              &lt;artifactId&gt;scala-library&lt;/artifactId&gt;
              &lt;version&gt;${scala.version}&lt;/version&gt;
          &lt;/dependency&gt;
  
          &lt;dependency&gt;
              &lt;groupId&gt;org.apache.spark&lt;/groupId&gt;
              &lt;artifactId&gt;spark-core_2.10&lt;/artifactId&gt;
              &lt;version&gt;${spark.version}&lt;/version&gt;
          &lt;/dependency&gt;
  &lt;dependency&gt;
              &lt;groupId&gt;org.apache.spark&lt;/groupId&gt;
              &lt;artifactId&gt;spark-hive_2.10&lt;/artifactId&gt;
              &lt;version&gt;${spark.version}&lt;/version&gt;
          &lt;/dependency&gt;
  
         &lt;dependency&gt;
         &lt;groupId&gt;org.apache.spark&lt;/groupId&gt;
         &lt;artifactId&gt;spark-streaming-kafka-0-10_2.10&lt;/artifactId&gt;
         &lt;version&gt;${spark.version}&lt;/version&gt;
         &lt;/dependency&gt;
  
          &lt;dependency&gt;
              &lt;groupId&gt;org.apache.spark&lt;/groupId&gt;
              &lt;artifactId&gt;spark-streaming_2.10&lt;/artifactId&gt;
              &lt;version&gt;${spark.version}&lt;/version&gt;
          &lt;/dependency&gt;
          &lt;dependency&gt;
              &lt;groupId&gt;org.apache.hadoop&lt;/groupId&gt;
              &lt;artifactId&gt;hadoop-hdfs&lt;/artifactId&gt;
              &lt;version&gt;2.7.2&lt;/version&gt;
          &lt;/dependency&gt;
          &lt;dependency&gt;
              &lt;groupId&gt;org.apache.hadoop&lt;/groupId&gt;
              &lt;artifactId&gt;hadoop-common&lt;/artifactId&gt;
              &lt;version&gt;2.7.2&lt;/version&gt;
          &lt;/dependency&gt;
          &lt;dependency&gt;
              &lt;groupId&gt;commons-io&lt;/groupId&gt;
              &lt;artifactId&gt;commons-io&lt;/artifactId&gt;
              &lt;version&gt;2.5&lt;/version&gt;
          &lt;/dependency&gt;
  &lt;/dependencies&gt;
  </code></pre>