title: Getting error : Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/SparkConf
body: <p>I am working on Kafka Spark Streaming. The IDLE doesn't show any errors and the program builds successfully as well but I am getting this error:</p>
  
  <pre><code>Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/SparkConf
  at KafkaSparkStream1$.main(KafkaSparkStream1.scala:13)
  at KafkaSparkStream1.main(KafkaSparkStream1.scala)
  Caused by: java.lang.ClassNotFoundException: org.apache.spark.SparkConf
  at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
  ... 2 more
  </code></pre>
  
  <p>I am using maven. I have also set up my environment variables correctly as every component is working individually My spark version is 3.0.0-preview2, Scala version is 2.12
  I have exported a spark-streaming-Kafka jar file. </p>
  
  <p>Here is my pom file:</p>
  
  <pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;
  &lt;project xmlns="http://maven.apache.org/POM/4.0.0"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"&gt;
  &lt;modelVersion&gt;4.0.0&lt;/modelVersion&gt;
  
  &lt;groupId&gt;com.org.cpg.casestudy&lt;/groupId&gt;
  &lt;artifactId&gt;Kafka_casestudy&lt;/artifactId&gt;
  &lt;version&gt;1.0-SNAPSHOT&lt;/version&gt;
  
  &lt;properties&gt;
      &lt;spark.version&gt;3.0.0&lt;/spark.version&gt;
      &lt;scala.version&gt;2.12&lt;/scala.version&gt;
  &lt;/properties&gt;
  
  &lt;build&gt;
      &lt;plugins&gt;
          &lt;!-- Maven Compiler Plugin--&gt;
          &lt;plugin&gt;
              &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
              &lt;artifactId&gt;maven-compiler-plugin&lt;/artifactId&gt;
              &lt;configuration&gt;
                  &lt;source&gt;1.8&lt;/source&gt;
                  &lt;target&gt;1.8&lt;/target&gt;
              &lt;/configuration&gt;
          &lt;/plugin&gt;
      &lt;/plugins&gt;
  &lt;/build&gt;
  
  &lt;dependencies&gt;
      &lt;!-- Apache Kafka Clients--&gt;
      &lt;dependency&gt;
          &lt;groupId&gt;org.apache.kafka&lt;/groupId&gt;
          &lt;artifactId&gt;kafka-clients&lt;/artifactId&gt;
          &lt;version&gt;2.5.0&lt;/version&gt;
      &lt;/dependency&gt;
      &lt;!-- Apache Kafka Streams--&gt;
      &lt;dependency&gt;
          &lt;groupId&gt;org.apache.kafka&lt;/groupId&gt;
          &lt;artifactId&gt;kafka-streams&lt;/artifactId&gt;
          &lt;version&gt;2.5.0&lt;/version&gt;
      &lt;/dependency&gt;
      &lt;!-- Apache Log4J2 binding for SLF4J --&gt;
      &lt;dependency&gt;
          &lt;groupId&gt;org.apache.logging.log4j&lt;/groupId&gt;
          &lt;artifactId&gt;log4j-slf4j-impl&lt;/artifactId&gt;
          &lt;version&gt;2.11.0&lt;/version&gt;
      &lt;/dependency&gt;
      &lt;!-- https://mvnrepository.com/artifact/org.apache.spark/spark-core --&gt;
      &lt;dependency&gt;
          &lt;groupId&gt;org.apache.spark&lt;/groupId&gt;
          &lt;artifactId&gt;spark-core_2.12&lt;/artifactId&gt;
          &lt;version&gt;3.0.0-preview2&lt;/version&gt;
          &lt;scope&gt;provided&lt;/scope&gt;
      &lt;/dependency&gt;
  
      &lt;!-- https://mvnrepository.com/artifact/org.apache.spark/spark-streaming --&gt;
      &lt;dependency&gt;
          &lt;groupId&gt;org.apache.spark&lt;/groupId&gt;
          &lt;artifactId&gt;spark-streaming_2.12&lt;/artifactId&gt;
          &lt;version&gt;3.0.0-preview2&lt;/version&gt;
          &lt;scope&gt;provided&lt;/scope&gt;
      &lt;/dependency&gt;
  
      &lt;!-- https://mvnrepository.com/artifact/org.apache.spark/spark-streaming-kafka-0-10 --&gt;
      &lt;dependency&gt;
          &lt;groupId&gt;org.apache.spark&lt;/groupId&gt;
          &lt;artifactId&gt;spark-streaming-kafka-0-10_2.12&lt;/artifactId&gt;
          &lt;version&gt;3.0.0-preview2&lt;/version&gt;
      &lt;/dependency&gt;
  
  &lt;/dependencies&gt;
  </code></pre>
  
  <p></p>
  
  <p>Here is my code (word count of message send by producer):</p>
  
  <pre><code>import org.apache.kafka.clients.consumer.ConsumerConfig
  import org.apache.spark._
  import org.apache.spark.streaming.kafka010.{ConsumerStrategies, KafkaUtils, LocationStrategies}
  import org.apache.spark.streaming.{Seconds, StreamingContext}
  import org.codehaus.jackson.map.deser.std.StringDeserializer
  
  object KafkaSparkStream {
  def main(args: Array[String]): Unit = {
  
  val brokers = "localhost:9092";
  val groupid = "GRP1";
  val topics = "KafkaTesting";
  val SparkConf = new SparkConf().setMaster("local[*]").setAppName("KafkaSparkStreaming");
  val ssc = new StreamingContext(SparkConf,Seconds(10))
  val sc = ssc.sparkContext
  
  sc.setLogLevel("off")
  
  val topicSet = topics.split(",").toSet
  
  val kafkaPramas = Map[String , Object](
    ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG -&gt; brokers,
     ConsumerConfig.GROUP_ID_CONFIG -&gt; groupid,
    ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG -&gt; classOf[StringDeserializer],
    ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG -&gt; classOf[StringDeserializer]
  )
  
  val messages = KafkaUtils.createDirectStream[String,String](
    ssc, LocationStrategies.PreferConsistent, ConsumerStrategies.Subscribe[String,String](topicSet,kafkaPramas)
  )
  
  val line=messages.map(_.value)
  val words = line.flatMap(_.split(" "))
  val wordCount = words.map(x=&gt; (x,1)).reduceByKey(_+_)
  
  wordCount.print()
  
  ssc.start()
  ssc.awaitTermination()
  
  }
  }
  </code></pre>