title: Apache Spark: Issues with Extracting Values from Row
body: <p>I've been having a lot of issues with the Row class in Spark. It seems to me Row class is one real badly designed class.  It should really be no more difficult to extract a value from a Row than extracting a value from a Scala list; but in practice, you have to know the exact type of the column in order to extract it.  You can't even turn the columns into String;  how ridiculous is that for a great framework such as Spark?  In real world, in most cases, you don't know the exact type of the column, and on top of that in many cases, you have dozens or hundreds of columns.  The following is an example to show you the ClassCastExceptions that I've been getting.  </p>
  
  <p>Does anyone have any solution for easy extraction of values from a Row?</p>
  
  <pre><code>scala&gt; val df = List((1,2),(3,4)).toDF("col1","col2")
  df: org.apache.spark.sql.DataFrame = [col1: int, col2: int]
  
  
  scala&gt; df.first.getAs[String]("col1")
  java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String
    ... 56 elided
  
  scala&gt; df.first.getAs[Int]("col1")
  res12: Int = 1
  
  scala&gt; df.first.getInt(0)
  res13: Int = 1
  
  scala&gt; df.first.getLong(0)
  java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long
    at scala.runtime.BoxesRunTime.unboxToLong(BoxesRunTime.java:105)
    at org.apache.spark.sql.Row$class.getLong(Row.scala:231)
    at org.apache.spark.sql.catalyst.expressions.GenericRow.getLong(rows.scala:165)
    ... 56 elided
  
  scala&gt; df.first.getFloat(0)
  java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Float
    at scala.runtime.BoxesRunTime.unboxToFloat(BoxesRunTime.java:109)
    at org.apache.spark.sql.Row$class.getFloat(Row.scala:240)
    at org.apache.spark.sql.catalyst.expressions.GenericRow.getFloat(rows.scala:165)
    ... 56 elided
  
  scala&gt; df.first.getString(0)
  java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String
    at org.apache.spark.sql.Row$class.getString(Row.scala:255)
    at org.apache.spark.sql.catalyst.expressions.GenericRow.getString(rows.scala:165)
    ... 56 elided 
  </code></pre>