title: How to remove the empty columns from dataframe of pyspark
body: <p><a href="https://github.com/1576dkm/name" rel="nofollow noreferrer">name data</a></p>
  
  <p>we have a dataframe:</p>
  
  <pre><code>names = spark.read.csv("name.csv", header="true", inferSchema="true").rdd
  </code></pre>
  
  <p>I want to do this:</p>
  
  <pre><code>res=names.filter(lambda f: f['Name'] == "Diwakar").map(lambda name: (name['Name'], name['Age']))
  res.toDF(['Name','Age']).write.csv("final", mode="overwrite", header="true")
  </code></pre>
  
  <p><strong>but empty column is creating the problem.</strong></p>