title: Pyspark agg function to "explode" rows into columns
body: <p>Basically, I have a dataframe that looks like this:</p>
  
  <pre><code>+----+-------+------+------+
  | id | index | col1 | col2 |
  +----+-------+------+------+
  | 1  | a     | a11  | a12  |
  +----+-------+------+------+
  | 1  | b     | b11  | b12  |
  +----+-------+------+------+
  | 2  | a     | a21  | a22  |
  +----+-------+------+------+
  | 2  | b     | b21  | b22  |
  +----+-------+------+------+
  </code></pre>
  
  <p>and my desired output is this:</p>
  
  <pre><code>+----+--------+--------+--------+--------+
  | id | col1_a | col1_b | col2_a | col2_b |
  +----+--------+--------+--------+--------+
  | 1  | a11    | b11    | a12    | b12    |
  +----+--------+--------+--------+--------+
  | 2  | a21    | b21    | a22    | b22    |
  +----+--------+--------+--------+--------+
  </code></pre>
  
  <p>So basically I want to "explode" the <code>index</code> column into new columns after I groupby <code>id</code>. Btw, the <code>id</code> counts are the same and each <code>id</code> has the same set of <code>index</code> values. I'm using pyspark.</p>