title: Combining values as dictionaries in pyspark
body: <p>I have data that looks like this:</p>
  
  <pre><code>main_data = sc.parallelize([
      ('key001', 'value001'),
      ('key001', 'value002'),
      ('key001', 'value001'),
      ('key002', 'value003'),
      ('key002', 'value003'),
      ('key002', 'value001'),
      ('key002', 'value004'),
      ('key003', 'value005'),
      ('key003', 'value006'),
  ])
  </code></pre>
  
  <p>And what I would like to have is a structure where every key contains a dictionary with the counts of every value, like:</p>
  
  <pre><code>('key001': {'value001': 2, 'value002': 1},
   'key002': {'value003': 2, 'value001': 1, 'value004': 1},
   'key003': {'value005': 1, 'value006': 1})
  </code></pre>
  
  <p>My initial idea was to first map the values of <code>main_data</code> to a <code>1</code> value so I could then sum the result, like:</p>
  
  <p><code>main_data.mapValues(lambda t: {t: 1})</code></p>
  
  <p>And after that I understand I should probably use <code>aggregateByKey</code>, but it's really escaping me how to do that.</p>