title: Exception in Pyspark Structured Streaming while reading from Kafka
body: <p><strong>Environment</strong>: Spark 2.4.0</p>
  
  <p>I have included spark-sql-kafka-0-10 jar, and it's of the same version as that of the Spark I am using.</p>
  
  <p>Here's the exception:</p>
  
  <pre><code>py4j.protocol.Py4JJavaError: An error occurred while calling o38.load.
  : java.lang.NoClassDefFoundError: org.apache.kafka.common.serialization.ByteArrayDeserializer
      at org.apache.spark.sql.kafka010.KafkaSourceProvider$.&lt;init&gt;(KafkaSourceProvider.scala:487)
      at org.apache.spark.sql.kafka010.KafkaSourceProvider$.&lt;clinit&gt;(KafkaSourceProvider.scala)
      at org.apache.spark.sql.kafka010.KafkaSourceProvider.validateStreamOptions(KafkaSourceProvider.scala:414)
      at org.apache.spark.sql.kafka010.KafkaSourceProvider.sourceSchema(KafkaSourceProvider.scala:66)
      at org.apache.spark.sql.execution.datasources.DataSource.sourceSchema(DataSource.scala:209)
      at org.apache.spark.sql.execution.datasources.DataSource.sourceInfo$lzycompute(DataSource.scala:95)
      at org.apache.spark.sql.execution.datasources.DataSource.sourceInfo(DataSource.scala:95)
      at org.apache.spark.sql.execution.streaming.StreamingRelation$.apply(StreamingRelation.scala:33)
      at org.apache.spark.sql.streaming.DataStreamReader.load(DataStreamReader.scala:171)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
      at java.lang.reflect.Method.invoke(Method.java:508)
      at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
      at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
      at py4j.Gateway.invoke(Gateway.java:282)
      at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
      at py4j.commands.CallCommand.execute(CallCommand.java:79)
      at py4j.GatewayConnection.run(GatewayConnection.java:238)
      at java.lang.Thread.run(Thread.java:812)
  Caused by: java.lang.ClassNotFoundException: org.apache.kafka.common.serialization.ByteArrayDeserializer
      at java.net.URLClassLoader.findClass(URLClassLoader.java:610)
      at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:937)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:882)
      at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:343)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:865)
      ... 20 more
  </code></pre>