title: Spark/Scala : Spark DataFrame column with Struct Type
body: <p>I have input like this .</p>
  
  <p><strong>Input:</strong> </p>
  
  <pre><code>|customerId|Header     |Line     |
  |1001      |1001aa     |1001aa1  |
  |1001      |1001aa     |1001aa2  |
  |1001      |1001aa     |1001aa3  |
  |1001      |1001aa     |1001aa4  |
  |1002      |1002bb     |1002bb1  |
  |1002      |1002bb     |1002bb2  |
  |1002      |1002bb     |1002bb3  |
  |1002      |1002bb     |1002bb4  |
  |1003      |1003cc     |1003cc1  |
  |1003      |1003cc     |1003cc2  |
  |1003      |1003cc     |1003cc3  |
  +----------+-----------+---------+
  </code></pre>
  
  <p>I want the output be of type<a href="https://i.stack.imgur.com/Q6vmU.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/Q6vmU.png" alt="enter image description here"></a></p>
  
  <p>Using Dataframe and UDF I am able to do this
  <a href="https://i.stack.imgur.com/t5rgE.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/t5rgE.png" alt="enter image description here"></a>
   But I would like to have those column names as well with struct Datatype. Any help is appreciated.</p>
  
  <pre><code> val udfHeaderLineList1 = udf((header:String,line:Seq[String])=&gt;{
    line.map(records=&gt;List(header,records)).toList
  })
  
  
  val eventingDFtable = my_dataframe_data_Table.
    groupBy(col("customerId"), col("header")).
    agg(collect_list(col("Line")).alias("Line")).
   withColumn("TransHeaderStruct",udfHeaderLineList1(col("header"),col("Line"))).printSchema
  </code></pre>