title: Spark UDF - Task not serializable exception
body: <p>I am trying to create UDF using the below scala code</p>
  
  <pre><code>lazy val formattedDF = df.withColumn("result_col", validateudf(df("id")))
  
  val validateudf = udf((id: Int) =&gt; {
  
    if(id == 1){
       "ID IS EQUAL TO 1"
    } 
    else if(id &gt; 1){
      validateId(id)
    }
    else{
      "NO VALID RECORDS"
    }
  })
  
  def validateId(id:Int) : String = {
     if (id &gt; 2) {
       "ID IS GREATER THAN 2"
     }
     else {
       "VALID RECORDS"
     }
   }
  </code></pre>
  
  <p>When I run this code, I am getting task not serializable exception.</p>
  
  <p>Any Ideas? Thanks.</p>