title: Convert GroupBy Object to Ordered List in Pyspark
body: <p>I'm using Spark 2.0.0 and dataframe.
  Here is my input dataframe as</p>
  
  
  
  <pre class="lang-none prettyprint-override"><code>| id | year        | qty    |
  |----|-------------|--------|
  | a  | 2012        | 10     |
  | b  | 2012        | 12     |
  | c  | 2013        | 5      |
  | b  | 2014        | 7      |
  | c  | 2012        | 3      |
  </code></pre>
  
  <p>What I want is </p>
  
  <pre class="lang-none prettyprint-override"><code>| id | year_2012 | year_2013 | year_2014 |
  |----|-----------|-----------|-----------|
  | a  | 10        | 0         | 0         |
  | b  | 12        | 0         | 7         |
  | c  | 3         | 5         | 0         |
  </code></pre>
  
  <p>or</p>
  
  <pre class="lang-none prettyprint-override"><code>| id | yearly_qty  |
  |----|---------------|
  | a  | [10, 0, 0]    |
  | b  | [12, 0, 7]    |
  | c  | [3, 5, 0]     |
  </code></pre>
  
  <p>The closest solution I found is <code>collect_list()</code> but this function doesn't provide order for the list. In my mind the solution should be like:</p>
  
  <pre class="lang-none prettyprint-override"><code>data.groupBy('id').agg(collect_function)
  </code></pre>
  
  <p>Is there a way to generate this without filtering every id out using a loop?</p>