title: Spark 1.3.1 Python ForeachRDD
body: <p>I am writing a script in python which takes an RDD from a kafka Dstream. I then want to take the RDD break it down and perform a hive query among other things.
  To do this I am using the following piece of code</p>
  
  <pre><code>recommendationDataset = parsedStream.map(lambda (v): (v.get("CustomerId"))).foreachRDD(generateRecommendation)
  </code></pre>
  
  <p>I then call a hive query from within generateRecommendation.
  Is it possible to perform a hive query in this command somehow without having to call foreachrdd and then a function? I'm knew to spark and python.</p>
  
  <pre><code>        def generateRecommendation(avroCustomer):
             print "=========================Generate Recommend========================="
             from pyspark import SparkContext,SparkConf
             from pyspark.sql import SQLContext
  
             print(type(avroCustomer))
             newAvroCustomer = avroCustomer.collect()
             for x in newAvroCustomer:
                print "===Query DB==="
                CustomerPurchaseHistorySingle=sqlContext.sql("SELECT * FROM CustomerPurchaseHistoryAllRec WHERE CustomerId=%s" % x)
                CustomerPurchaseHistorySingle.show()
                print(type(CustomerPurchaseHistorySingle))
                print(type(CustomerPurchaseHistoryAllRec))
  
  
             print "=======================Generated Recommend!========================"
  </code></pre>