title: To read a field with comma and quotes in csv where comma is delimiter - pyspark
body: <p>I have a record in my input csv file as,</p>
  
  <pre><code>"2017-11-01","2017-10-29","2017-11-04","4532491","","","","Natural States: "The Environmental Imagination" in Maine, Oregon, and the Nation","1000","Richard W. Judd"
  </code></pre>
  
  <p>When I read this csv in pyspark, the field <code>"Natural States: "The Environmental Imagination" in Maine, Oregon, and the Nation"</code> gets delimited as separate columns.</p>
  
  <pre><code>&gt;&gt;&gt; df = spark.read.csv('file.csv')
  &gt;&gt;&gt; df.show(truncate=False)
  +----------+----------+----------+----------+----+----+----+---------------------------------------------------------+-------+----------------+----+---------------+
  |_c0       |_c1       |_c2       |_c3       |_c4 |_c5 |_c6 |_c7                                                      |_c8    |_c9             |_c10|_c11           |
  +----------+----------+----------+----------+----+----+----+---------------------------------------------------------+-------+----------------+----+---------------+
  |2017-11-01|2017-10-29|2017-11-04| 4532491  |null|null|null|Natural States: "The Environmental Imagination" in Maine | Oregon| and the Nation |1000|Richard W. Judd|
  +----------+----------+----------+----------+----+----+----+---------------------------------------------------------+-------+----------------+----+---------------+
  </code></pre>
  
  <p>Any workaround apart from changing the delimiter in input file, as we can't change the input file.</p>