title: Spark SQL execution fails with error: java.lang.NoClassDefFoundError: org.codehaus.janino.InternalCompilerException
body: <p>Running a Spark SQL program in Java immediately fails with the following exception, as soon as the first action is called on a dataset. Have tried all the suggestions in <a href="https://stackoverflow.com/questions/42352091/spark-sql-fails-with-java-lang-noclassdeffounderror-org-codehaus-commons-compil">Spark SQL fails with java.lang.NoClassDefFoundError: org/codehaus/commons/compiler/UncheckedCompileException</a> nothing seems to work. Tried upgrading versions of Spark still facing the same error.</p>
  <p>Please note not running using <code>spark-submit</code> but via <code>java -jar &lt;app-name&gt;</code>
  Below is Spark Gradle config:</p>
  <pre><code>    compile group: 'org.apache.spark', name: 'spark-sql_2.12', version: '2.4.3'
      implementation 'org.codehaus.janino:commons-compiler:3.0.16'
      implementation 'org.codehaus.janino:janino:3.0.16'
  </code></pre>
  <p>Tried below exclusion config too, still same error persists:</p>
  <pre><code>    implementation('org.apache.spark:spark-sql_2.12:2.4.3') {
          exclude group: 'org.codehaus.janino', module: 'janino'
          exclude group: 'org.codehaus.janino', module: 'commons-compiler'
      }
      compile &quot;org.codehaus.janino:commons-compiler:3.0.16&quot;
      compile &quot;org.codehaus.janino:janino:3.0.16&quot;
  </code></pre>
  <p>Exception stack trace:</p>
  <pre><code>java.lang.NoClassDefFoundError: org.codehaus.janino.InternalCompilerException
      at org.apache.spark.sql.catalyst.expressions.codegen.JavaCode$.variable(javaCode.scala:63)
      at org.apache.spark.sql.catalyst.expressions.codegen.JavaCode$.isNullVariable(javaCode.scala:76)
      at org.apache.spark.sql.catalyst.expressions.Expression.$anonfun$genCode$3(Expression.scala:109)
      at org.apache.spark.sql.catalyst.expressions.Expression$$Lambda$2984/0x00000000bfa2b020.apply(Unknown Source)
      at scala.Option.getOrElse(Option.scala:189)
      at org.apache.spark.sql.catalyst.expressions.Expression.genCode(Expression.scala:105)
      at org.apache.spark.sql.catalyst.expressions.codegen.GenerateSafeProjection$.$anonfun$create$1(GenerateSafeProjection.scala:155)
      at org.apache.spark.sql.catalyst.expressions.codegen.GenerateSafeProjection$$$Lambda$2982/0x00000000ef9c1620.apply(Unknown Source)
      at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:238)
      at scala.collection.TraversableLike$$Lambda$1442/0x00000000ff849e20.apply(Unknown Source)
      at scala.collection.immutable.List.foreach(List.scala:392)
      at scala.collection.TraversableLike.map(TraversableLike.scala:238)
      at scala.collection.TraversableLike.map$(TraversableLike.scala:231)
      at scala.collection.immutable.List.map(List.scala:298)
      at org.apache.spark.sql.catalyst.expressions.codegen.GenerateSafeProjection$.create(GenerateSafeProjection.scala:152)
      at org.apache.spark.sql.catalyst.expressions.codegen.GenerateSafeProjection$.create(GenerateSafeProjection.scala:38)
      at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator.generate(CodeGenerator.scala:1193)
      at org.apache.spark.sql.Dataset.collectFromPlan(Dataset.scala:3382)
      at org.apache.spark.sql.Dataset.$anonfun$collectAsList$1(Dataset.scala:2794)
      at org.apache.spark.sql.Dataset$$Lambda$2827/0x00000000af983620.apply(Unknown Source)
      at org.apache.spark.sql.Dataset.$anonfun$withAction$2(Dataset.scala:3364)
      at org.apache.spark.sql.Dataset$$Lambda$2919/0x00000000cf9f4220.apply(Unknown Source)
      at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:78)
      at org.apache.spark.sql.execution.SQLExecution$$$Lambda$2920/0x00000000cfa4f020.apply(Unknown Source)
      at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:125)
      at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:73)
      at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3364)
      at org.apache.spark.sql.Dataset.collectAsList(Dataset.scala:2793)
  </code></pre>
  <p>Code implementation:</p>
  <pre><code>SparkConf sparkConf = new SparkConf().setAppName(&quot;App demo&quot;).setMaster(&quot;local[*]&quot;);
  
  try (SparkSession sparkSession = createSparkSession(sparkConf)) {
      Dataset&lt;Row&gt; df = sparkSession.read().json(&quot;/Users/shubhampr/Documents/spark/examples/src/main/resources/people.json&quot;);
      df.show();
  } catch (Exception e) {
      log.error(&quot;Error in processing file: {}&quot;, e.getMessage());
              return;
  }
  
  SparkSession createSparkSession(SparkConf sparkConf) {
          return SparkSession.builder()
                  .sparkContext(new JavaSparkContext(sparkConf).sc())
                  .getOrCreate();
  }
  </code></pre>