I am new to pyspark . My requirement is to get/extract the attribute names from a nested json file . I tried using json\_normalize imported from pandas package. It works for direct attributes but never fetches the attributes within json array attributes. My json doesn't have a static structure. It varies for each document that we receive. Could someone please help me with explanation for the small example provided below,

```
        {  
               "id":"1",
               "name":"a",
               "salaries":[  
                  {  
                     "salary":"1000"
                  },
                  {  
                     "salary":"5000"
                  }
               ],
               "states":{  
                  "state":"Karnataka",
                  "cities":[  
                     {  
                        "city":"Bangalore"
                     },
                     {  
                        "city":"Mysore"
                     }
                  ],
                  "state":"Tamil Nadu",
                  "cities":[  
                     {  
                        "city":"Chennai"
                     },
                     {  
                        "city":"Coimbatore"
                     }
                  ]
               }
            }  

```

Especially for the json array elements..

Expected output : 
id 
name
salaries.salary
states.state
states.cities.city``