title: how to convert rows into columns in spark dataframe, scala
body: <p>Is there any way to transpose dataframe rows into columns.
  I have following structure as a input:</p>
  
  <pre><code>val inputDF = Seq(("pid1","enc1", "bat"),
                    ("pid1","enc2", ""),
                    ("pid1","enc3", ""),
                    ("pid3","enc1", "cat"),
                    ("pid3","enc2", "")
                ).toDF("MemberID", "EncounterID", "entry" )
  
  inputDF.show:
  
  +--------+-----------+-----+
  |MemberID|EncounterID|entry|
  +--------+-----------+-----+
  |    pid1|       enc1|  bat|
  |    pid1|       enc2|     |
  |    pid1|       enc3|     |
  |    pid3|       enc1|  cat|
  |    pid3|       enc2|     |
  +--------+-----------+-----+
  
  expected result:
  
  +--------+----------+----------+----------+-----+
  |MemberID|Encounter1|Encounter2|Encounter3|entry|
  +--------+----------+----------+----------+-----+
  |    pid1|      enc1|      enc2|      enc3|  bat|
  |    pid3|      enc1|      enc2|      null|  cat|
  +--------+----------+----------+----------+-----+
  </code></pre>
  
  <p>Please suggest if there is any optimized direct API available for transposing rows into columns.
  my input data size is quite huge, so actions like collect, I wont be able to perform as it would take all the data on driver.
  I am using Spark 2.x</p>