title: How could I unpivot a dataframe in Spark?
body: <p>I have a dataframe with the following schema:</p>
  
  <pre><code>subjectID, feature001, feature002, feature003, ..., feature299
  </code></pre>
  
  <p>Let's say my dataframe looks like:</p>
  
  <pre><code>123,0.23,0.54,0.35,...,0.26   
  234,0.17,0.49,0.47,...,0.69
  </code></pre>
  
  <p>Now, what I want is:</p>
  
  <pre><code>subjectID, featureID, featureValue
  </code></pre>
  
  <p>The above dataframe would look like:</p>
  
  <pre><code>123,001,0.23
  123,002,0.54
  123,003,0.35
  ......
  123,299,0.26
  234,001,0.17
  234,002,0.49
  234,003,0.47
  ......
  234,299,0.69
  </code></pre>
  
  <p>I know how to achieve it if i have only several columns:</p>
  
  <pre><code>newDF = df.select($"subjectID", expr("stack(3, 'feature001', 001, 'feature002', 002, 'feature003', 003) as (featureID, featureValue)"))
  </code></pre>
  
  <p>However, I am looking for a way to deal with 300 columns.</p>