title: Unable to find LoginModule class: org.apache.kafka.common.security.plain.PlainLoginModule
body: <p>Environment: Spark 2.3.0, Scala 2.11.12, Kafka (Whatever the latest version is)</p>
  
  <p>I have a secure Kafka system, to which I'm trying to connect my Spark Streaming Consumer. Below is my <code>build.sbt</code> file:</p>
  
  <pre><code>name := "kafka-streaming"
  version := "1.0"
  
  scalaVersion := "2.11.12"
  
  // still want to be able to run in sbt
  // https://github.com/sbt/sbt-assembly#-provided-configuration
  run in Compile &lt;&lt;= Defaults.runTask(fullClasspath in Compile, mainClass in (Compile, run), runner in (Compile, run))
  
  fork in run := true
  javaOptions in run ++= Seq(
      "-Dlog4j.debug=true",
      "-Dlog4j.configuration=log4j.properties")
  
  assemblyMergeStrategy in assembly := {
      case "META-INF/services/org.apache.spark.sql.sources.DataSourceRegister" =&gt; MergeStrategy.concat
      case PathList("META-INF", _*) =&gt; MergeStrategy.discard
      case _ =&gt; MergeStrategy.first
  }
  
  libraryDependencies ++= Seq(
      "org.apache.spark" %% "spark-core" % "2.3.0",
      "org.apache.spark" %% "spark-sql" % "2.3.0",
      "org.apache.spark" %% "spark-streaming" % "2.3.0",
      "org.apache.spark" %% "spark-streaming-kafka-0-10" % "2.3.0",
      "org.apache.spark" %% "spark-sql-kafka-0-10" % "2.3.0",
      "com.ibm.db2.jcc" % "db2jcc" % "db2jcc4"
  )
  </code></pre>
  
  <p>Note that this is Spark 2.3.0, and I cannot change my Spark version.</p>
  
  <p>Now here's my part of the code where I try to connect my Spark Streaming consumer to my secured Kafka:</p>
  
  <pre class="lang-scala prettyprint-override"><code>val df = spark.readStream
              .format("kafka")
              .option("subscribe", "raw_weather")
              .option("kafka.bootstrap.servers", "&lt;url:port&gt;s")
              .option("kafka.security.protocol", "SASL_SSL")
              .option("kafka.sasl.mechanism" , "PLAIN")
              .option("kafka.sasl.jaas.config", "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"user\" password=\"" + "password" + "\";")
              .option("kafka.ssl.protocol", "TLSv1.2")
              .option("kafka.ssl.enabled.protocols", "TLSv1.2")
              .option("kafka.ssl.endpoint.identification.algorithm", "HTTPS")
              .load()
  </code></pre>
  
  <p>When I try to run this program, the following error is thrown:</p>
  
  <pre><code>Exception in thread "main" org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
      at org.apache.kafka.clients.consumer.KafkaConsumer.&lt;init&gt;(KafkaConsumer.java:702)
      at org.apache.kafka.clients.consumer.KafkaConsumer.&lt;init&gt;(KafkaConsumer.java:557)
      at org.apache.kafka.clients.consumer.KafkaConsumer.&lt;init&gt;(KafkaConsumer.java:540)
      at org.apache.spark.sql.kafka010.SubscribeStrategy.createConsumer(ConsumerStrategy.scala:62)
      at org.apache.spark.sql.kafka010.KafkaOffsetReader.createConsumer(KafkaOffsetReader.scala:314)
      at org.apache.spark.sql.kafka010.KafkaOffsetReader.&lt;init&gt;(KafkaOffsetReader.scala:78)
      at org.apache.spark.sql.kafka010.KafkaSourceProvider.createContinuousReader(KafkaSourceProvider.scala:130)
      at org.apache.spark.sql.kafka010.KafkaSourceProvider.createContinuousReader(KafkaSourceProvider.scala:43)
      at org.apache.spark.sql.streaming.DataStreamReader.load(DataStreamReader.scala:185)
      &gt;&gt; at com.ibm.kafkasparkintegration.executables.WeatherDataStream$.getRawDataFrame(WeatherDataStream.scala:74)
      at com.ibm.kafkasparkintegration.executables.WeatherDataStream$.main(WeatherDataStream.scala:24)
      at com.ibm.kafkasparkintegration.executables.WeatherDataStream.main(WeatherDataStream.scala)
  Caused by: org.apache.kafka.common.KafkaException: javax.security.auth.login.LoginException: unable to find LoginModule class:  org.apache.kafka.common.security.plain.PlainLoginModule
      at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:86)
      at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:70)
      at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:83)
      at org.apache.kafka.clients.consumer.KafkaConsumer.&lt;init&gt;(KafkaConsumer.java:623)
      ... 11 more
  Caused by: javax.security.auth.login.LoginException: unable to find LoginModule class:  org.apache.kafka.common.security.plain.PlainLoginModule
      at javax.security.auth.login.LoginContext.invoke(LoginContext.java:794)
      at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
      at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
      at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
      at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
      at org.apache.kafka.common.security.authenticator.AbstractLogin.login(AbstractLogin.java:69)
      at org.apache.kafka.common.security.authenticator.LoginManager.&lt;init&gt;(LoginManager.java:46)
      at org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:68)
      at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:78)
      ... 14 more
  </code></pre>
  
  <p>The <code>&gt;&gt;</code> in the error logs points to <code>load()</code> from the snippet above. I've been trying to figure this for a few days now, but haven't had much success.</p>