title: PySpark: custom function in aggregation on grouped data
body: <p>I have a PySpark DataFrame containing things as</p>
  
  <pre><code>Row(id='id1', type='A', status='H', keywords=['k1', 'k2', 'k3'])
  </code></pre>
  
  <p>Status is a binary option ('S'/'H').
  what I need to do is counting the ratio of occurrences in status <em>S</em> per each keyword per type, id and status. 
  Ratio will be</p>
  
  <pre><code>s/(s+h)
  </code></pre>
  
  <p>where <em>s</em> and <em>h</em> here are the occurrences.
  So for instance, if keyword <em>k1</em> occurs 2 times as S and 3 times as H in type <em>A</em> I'll want 2/3 for it in that type and my final output would ideally be </p>
  
  <pre><code>Row(id='id1', type='A', keyword='k1', ratio=0.66)
  </code></pre>
  
  <p>I was thinking this has to pass through several steps, and I'd be happy with computing the occurrences in S and H and then creating further column to ratio the two.</p>
  
  <p>But how would I compute the said occurrences after I run a <em>groupBy</em> by 'id', 'type' and 'status'? Would there be a way to run an <em>agg</em> with a custom function?</p>