title: Write results from Kafka to csv in pyspark
body: <p>I have setup a Kafka broker and I manage to read the records with pyspark.</p>
  
  <pre><code>import os
  from pyspark.sql import SparkSession
  import pyspark
  import sys
  from pyspark import SparkConf, SparkContext, SQLContext, Row
  from pyspark.streaming import StreamingContext
  from pyspark.streaming.kafka import KafkaUtils
  
  conf = SparkConf().setMaster("my-master").setAppName("Kafka_Spark")
  sc = SparkContext(conf=conf)
  sc.setLogLevel("WARN")
  ssc = StreamingContext(sc,5)
  kvs = KafkaUtils.createDirectStream(ssc,
                                          ['enriched_messages'], 
                                         {"metadata.broker.list":"my-kafka-broker","auto.offset.reset" : "smallest"},
                                          keyDecoder=lambda x: x,
                                          valueDecoder=lambda x: x)
  
  
  lines  = kvs.map(lambda x: x[1])
  lines.pprint()
  
  ssc.start()
  ssc.awaitTermination(10)
  </code></pre>
  
  <p>Example of returning data (timestamp, name, lastname, height):</p>
  
  <pre><code>2020-05-07 09:16:38, JoHN, Doe, 182.5
  </code></pre>
  
  <p>I want to write these records into a csv file. <code>lines</code> is of type <code>KafkaTransformedDStream</code> and classic solution with rdd is not working.
  Has anyone a solution to this? </p>