title: Read a csv into an RDD using Spark 2.0
body: <p>I want to read a csv file into a RDD using Spark 2.0. I can read it into a dataframe using</p>
  
  <pre><code>df = session.read.csv("myCSV.csv", header=True,)
  </code></pre>
  
  <p>and I can load it as a text file and then process it using</p>
  
  <pre><code>import csv
  rdd = context.textFile("myCSV.csv")
  header = rdd.first().replace('"','').split(',')
  rdd = (rdd.mapPartitionsWithIndex(lambda idx, itr: itr.drop(1) if idx == 0 else itr)
          .mapPartitions(lambda x: csv.reader(x))
        )
  </code></pre>
  
  <p>Is there a way to use the built in csv reader (spark-csv) to go straight to an RDD without having to convert from a dataframe to a csv? Or maybe the above RDD method is good enough as the built in reader does something similar under the hood? </p>
  
  <p><strong>Edit</strong>: 1) Again, I don't want to read into a dataframe and then convert to RDD. This will build up an entire structure only to have it immediately dropped. Seems pointless. 2) Yes, I can time the above (against DF -> RDD conversion), but that will only tell me if my RDD read solution is better than conversion. A built in csv to RDD method will most likely be more optimized than the above code. </p>