title: Pyspark Pair RDD from Text File
body: <p>I have a local text file <code>kv_pair.log</code> formatted such as that key value pairs are comma delimited and records begin and terminate with a new line:</p>
  
  <pre><code>"A"="foo","B"="bar","C"="baz"
  "A"="oof","B"="rab","C"="zab"
  "A"="aaa","B"="bbb","C"="zzz"
  </code></pre>
  
  <p>I am trying to read this to a Pair RDD using pySpark as follows:</p>
  
  <pre><code>from pyspark import SparkContext
  sc=sparkContext()
  
  # Read raw text to RDD
  lines=sc.textFile('kv_pair.log')
  
  # How to turn this into a Pair RDD?
  pairs=lines.map(lambda x: (x.replace('"', '').split(",")))
  
  print type(pairs)
  print pairs.take(2)
  </code></pre>
  
  <p>I feel I am close! The output of above is:</p>
  
  <pre><code>[[u'A=foo', u'B=bar', u'C=baz'], [u'A=oof', u'B=rab', u'C=zab']]
  </code></pre>
  
  <p>So it looks like <code>pairs</code> is a list of records, which contains a list of the kv pairs as strings.</p>
  
  <p>How can I use pySpark to transform this into a Pair RDD such as that the keys and values are properly separated?</p>
  
  <p>Ultimate goal is to transform <strong>this Pair RDD into a DataFrame to perform SQL operations</strong> - but one step at a time, please help transforming this into a Pair RDD.</p>