title: How to read file in pyspark with "]|[" delimiter
body: <p>The data looks like this:</p>
  
  <p><code>pageId]|[page]|[Position]|[sysId]|[carId
   0005]|[bmw]|[south]|[AD6]|[OP4</code></p>
  
  <p>There are atleast 50 columns and millions of rows. </p>
  
  <p>I did try to use below code to read:</p>
  
  <p><code>dff = sqlContext.read.format("com.databricks.spark.csv").option("header", "true").option("inferSchema", "true").option("delimiter", "]|[").load(trainingdata+"part-00000")</code></p>
  
  <p>it gives me following error:</p>
  
  <p><code>IllegalArgumentException: u'Delimiter cannot be more than one character: ]|['</code></p>