title: Spark: UDF executed many times
body: <p>I have a dataframe with the following code:</p>
  
  <pre><code>def test(lat: Double, lon: Double) = {
    println(s"testing ${lat / lon}")
    Map("one" -&gt; "one", "two" -&gt; "two")
  }
  
  val testUDF = udf(test _)
  
  df.withColumn("test", testUDF(col("lat"), col("lon")))
    .withColumn("test1", col("test.one"))
    .withColumn("test2", col("test.two"))
  </code></pre>
  
  <p>Now checking the logs, I found out that  for each row the UDF is executed 3 times. If I add the "test3" from a "test.three" column then the UDF is executed once more.</p>
  
  <p>Can someone explain me why?</p>
  
  <p>Can this be avoid properly (without caching the dataframe after "test" is added, even if this works)?</p>