title: storing a Dataframe to a hive partition table in spark
body: <p>I'm trying to store a stream of data comming in from a kafka topic into a hive partition table. I was able to convert the dstream to a dataframe and created a hive context. My code looks like this</p>
  
  <pre><code>val hiveContext = new HiveContext(sc)
  hiveContext.setConf("hive.exec.dynamic.partition", "true")
  hiveContext.setConf("hive.exec.dynamic.partition.mode", "nonstrict")
  newdf.registerTempTable("temp") //newdf is my dataframe
  newdf.write.mode(SaveMode.Append).format("osv").partitionBy("date").saveAsTable("mytablename")
  </code></pre>
  
  <p>But when I deploy the app on cluster, its says</p>
  
  <pre><code>Exception in thread "main" java.lang.IllegalArgumentException: Wrong FS: file:/tmp/spark-3f00838b-c5d9-4a9a-9818-11fbb0007076/scratch_hive_2016-10-18_23-18-33_118_769650074381029645-1, expected: hdfs://
  </code></pre>
  
  <p>When I try to save it as a normal table and comment out the  hiveconfigurations it work. But, with partition table...its giving me this error.</p>
  
  <p>I also tried registering the dataframe as a temp table and then to write that table to the partition table. Doing that also gave me the same error</p>
  
  <p>Can someone please tell how can I solve it.
  Thanks.</p>