title: Spark-shell : The number of columns doesn't match
body: <p>I have csv format file and is separated by delimiter pipe "|". And the dataset has 2 column, like below .</p>
  
  <pre><code>Column1|Column2
  1|Name_a
  2|Name_b
  </code></pre>
  
  <p>But sometimes we receive only one column value and other is missing like below</p>
  
  <pre><code>Column1|Column2
  1|Name_a
  2|Name_b
  3
  4
  5|Name_c
  6
  7|Name_f
  </code></pre>
  
  <p>So any row having mismatched column no is a garbage value for us for the above example it will be rows having column value as <code>3, 4, and 6</code> and we want to discard these rows. Is there any direct way I can discard those rows, without having a exception while reading the data from spark-shell like below.</p>
  
  <pre><code>val readFile = spark.read.option("delimiter", "|").csv("File.csv").toDF(Seq("Column1", "Column2"): _*)
  </code></pre>
  
  <p>When we are trying to read the file we are getting the below exception.</p>
  
  <pre><code>java.lang.IllegalArgumentException: requirement failed: The number of columns doesn't match.
  Old column names (1): _c0
  New column names (2): Column1, Column2
    at scala.Predef$.require(Predef.scala:224)
    at org.apache.spark.sql.Dataset.toDF(Dataset.scala:435)
    ... 49 elided 
  </code></pre>