title: Spark dataframe: Schema for type Unit is not supported
body: <p>I am using Spark 1.5.0 and I have this issue:</p>
  
  <pre><code>val df = paired_rdd.reduceByKey { 
      case (val1, val2) =&gt; val1 + "|" + val2 
  }.toDF("user_id","description")
  </code></pre>
  
  <p>Here is sample data for df, as you can see the column description has
  this format <code>(text1#text3#weight | text1#text3#weight|....)</code></p>
  
  <p>user1</p>
  
  <blockquote>
    <p>book1#author1#0.07841217886795074|tool1#desc1#0.27044260397331488|song1#album1#-0.052661673730870676|item1#category1#-0.005683148395350108</p>
  </blockquote>
  
  <p>I want to sort this df based on weight in descending order here is what I tried: </p>
  
  <p>First split the contents at "|" and then for each of those strings, split them at "#" and get the 3rd string which is weight and then convert that into a double value</p>
  
  <pre><code>val getSplitAtWeight = udf((str: String) =&gt; { 
      str.split("|").foreach(_.split("#")(2).toDouble)
  })
  </code></pre>
  
  <p>Sort based on the weigh value returned by the udf (in descending manner)</p>
  
  <pre><code>val df_sorted = df.sort(getSplitAtWeight(col("description")).desc)
  </code></pre>
  
  <p>I get the following error: </p>
  
  <blockquote>
    <p>Exception in thread "main" java.lang.UnsupportedOperationException:
    Schema for type Unit is not supported at
    org.apache.spark.sql.catalyst.ScalaReflection$class.schemaFor(ScalaReflection.scala:153)
    at
    org.apache.spark.sql.catalyst.ScalaReflection$.schemaFor(ScalaReflection.scala:29)
    at
    org.apache.spark.sql.catalyst.ScalaReflection$class.schemaFor(ScalaReflection.scala:64)
    at
    org.apache.spark.sql.catalyst.ScalaReflection$.schemaFor(ScalaReflection.scala:29)
    at org.apache.spark.sql.functions$.udf(functions.scala:2242)</p>
  </blockquote>