spark 1.3.1: Dataframe breaking MLib API
I think community is going to patch this. But for now, we can use Dataframe.rdd in ALS.train (or any other place where we see only RDDs are allowed)
I am trying to use Spark SQL and MLib together to create a recommendation program (extending movie recommendation program) in python. It was working fine with 1.2.0.
However, in 1.3.1, by default spark create Dataframe objects instead of SchemaRDD objects as output of a SQL. hence, mlib.ALS.train method is failing with an assertion error:
assert(ratings,RDD) 
(of course ratings is not RDD anymore :) )
Anyone facing this issue? Any workaround (I am thinking to use a map just to convert DF to RDD, but thats stupid :) )