title: JSON file parsing in Pyspark
body: <p>I am very new to Pyspark. I tried parsing the JSON file using the following code</p>
  
  <pre><code>from pyspark.sql import SQLContext
  sqlContext = SQLContext(sc)
  df = sqlContext.read.json("file:///home/malwarehunter/Downloads/122116-path.json")
  df.printSchema()
  </code></pre>
  
  <p>The output is as follows.</p>
  
  <blockquote>
    <p>root
    |-- _corrupt_record: string (nullable = true)</p>
  </blockquote>
  
  <pre><code>df.show()
  </code></pre>
  
  <p>The output looks like this </p>
  
  <pre><code>+--------------------+
  |     _corrupt_record|
  +--------------------+
  |                   {|
  |  "time1":"2...|
  |  "time2":"201...|
  |    "step":0.5,|
  |          "xyz":[|
  |                   {|
  |      "student":"00010...|
  |      "attr...|
  |        [ -2.52, ...|
  |        [ -2.3, -...|
  |        [ -1.97, ...|
  |        [ -1.27, ...|
  |        [ -1.03, ...|
  |        [ -0.8, -...|
  |        [ -0.13, ...|
  |        [ 0.09, -...|
  |        [ 0.54, -...|
  |        [  1.1, -...|
  |        [ 1.34, 0...|
  |        [ 1.64, 0...|
  +--------------------+
  only showing top 20 rows
  </code></pre>
  
  <p>The Json File looks like this.</p>
  
  <pre><code>{
    "time1":"2016-12-16T00:00:00.000",
  
    "time2":"2016-12-16T23:59:59.000",
  
    "step":0.5,
  
     "xyz":[
  
      {
       "student":"0001025D0007F5DB",
        "attr":[
      [ -2.52, -1.17 ],
      [ -2.3, -1.15 ],
      [ -1.97, -1.19 ],
      [ 10.16, 4.08 ],
      [ 10.23, 4.87 ],
      [ 9.96, 5.09 ] ]
  },
  {
    "student":"0001025D0007F5DC",
    "attr":[
      [ -2.58, -0.99 ],
      [ 10.12, 3.89 ],
      [ 10.27, 4.59 ],
      [ 10.05, 5.02 ] ]
  }
  ]}
  </code></pre>
  
  <p>Could you help me on parsing this and creating a Data Frame like this. </p>
  
  <p><a href="https://i.stack.imgur.com/dEc1E.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/dEc1E.jpg" alt="Output Dataframe required"></a></p>