title: Scala and Spark UDF function
body: <p>I made a simple UDF to convert or extract some values from a time field in a temptabl in spark. I register the function but when I call the function using sql it throws a NullPointerException. Below is my function and process of executing it. I am using Zeppelin. Strangly this was working yesterday but it stopped working this morning. </p>
  
  <p>Function</p>
  
  <pre><code>def convert( time:String ) : String = {
    val sdf = new java.text.SimpleDateFormat("HH:mm")
    val time1 = sdf.parse(time)
    return sdf.format(time1)
  }
  </code></pre>
  
  <p>Register the Function</p>
  
  <pre><code>sqlContext.udf.register("convert",convert _)
  </code></pre>
  
  <p>Test the function without SQL -- This works</p>
  
  <pre><code>convert(12:12:12) -&gt; returns 12:12
  </code></pre>
  
  <p>Test the function with SQL in Zeppelin this FAILS. </p>
  
  <pre><code>%sql
  select convert(time) from temptable limit 10
  </code></pre>
  
  <p>Structure of temptable</p>
  
  <pre><code>root
   |-- date: string (nullable = true)
   |-- time: string (nullable = true)
   |-- serverip: string (nullable = true)
   |-- request: string (nullable = true)
   |-- resource: string (nullable = true)
   |-- protocol: integer (nullable = true)
   |-- sourceip: string (nullable = true)
  </code></pre>
  
  <p>Part of the stacktrace that I am getting. </p>
  
  <pre><code>java.lang.NullPointerException
      at org.apache.hadoop.hive.ql.exec.FunctionRegistry.getFunctionInfo(FunctionRegistry.java:643)
      at org.apache.hadoop.hive.ql.exec.FunctionRegistry.getFunctionInfo(FunctionRegistry.java:652)
      at org.apache.spark.sql.hive.HiveFunctionRegistry.lookupFunction(hiveUdfs.scala:54)
      at org.apache.spark.sql.hive.HiveContext$$anon$3.org$apache$spark$sql$catalyst$analysis$OverrideFunctionRegistry$$super$lookupFunction(HiveContext.scala:376)
      at org.apache.spark.sql.catalyst.analysis.OverrideFunctionRegistry$$anonfun$lookupFunction$2.apply(FunctionRegistry.scala:44)
      at org.apache.spark.sql.catalyst.analysis.OverrideFunctionRegistry$$anonfun$lookupFunction$2.apply(FunctionRegistry.scala:44)
      at scala.Option.getOrElse(Option.scala:120)
      at org.apache.spark.sql.catalyst.analysis.OverrideFunctionRegistry$class.lookupFunction(FunctionRegistry.scala:44)
  </code></pre>