title: Spark Dataframe - How to get a particular field from a struct type column
body: <p>I have a data frame with a structure like this:</p>
  
  <pre class="lang-none prettyprint-override"><code>root
   |-- npaDetails: struct (nullable = true)
   |    |-- additionalInformation: struct (nullable = true)
   |    |-- npaStatus: struct (nullable = true)
   |    |-- npaDetails: struct (nullable = true)
   |-- npaHeaderData: struct (nullable = true)
   |    |-- npaNumber: string (nullable = true)
   |    |-- npaDownloadDate: string (nullable = true)     
   |    |-- npaDownloadTime: string (nullable = true) 
  </code></pre>
  
  <p>I want to retrieve all <code>npaNumber</code> from all the rows in the dataframe.</p>
  
  <p>My approach was to iterate over all rows in the data frame to extract for each one the value stored in the column <code>npaHeaderData</code>, in the field <code>npaNumber</code>. So I code the following lines:</p>
  
  <pre><code>parquetFileDF.foreach { newRow =&gt;  
  
    //To retrieve the second column
    val column = newRow.get(1)
  
    //The following line is not allowed
    //val npaNumber= column.getAs[String]("npaNumber")  
  
    println(column)
  
  }
  </code></pre>
  
  <p>The content of column printed in each iteration looks like:</p>
  
  <blockquote>
    <p>[207400956,27FEB17,09.30.00]</p>
  </blockquote>
  
  <p>But <code>column</code> is of type Any and I am not able extract any of its fields. Can anyone tell what am I doing wrong or what approach should I follow instead of this?</p>
  
  <p>Thanks</p>