title: Python multiprocessing for each key in dictionary
body: <p>I am new to python and i am trying to scale my processing in parallel. I have a file with certain number of tuples, each with certain value in the last column. I want to split this file data and apply my function in parallel to each chunk. But the thing is to split the data in to chunks based on last column value and apply the function for each chunk. For example, last column may have 'a' for some tuples and 'b' for some and 'c' for some. So in that case, i should get three chunks and process it in parallel. Number of unique values in last column may change depends on dataset, so i need to use the CPU accordingly.</p>
  
  <p>Q1:
  What i tried till now is to read the file and created a dictionary based on that records, so basically three key-value pairs for the above one, one with 'a' as key and all records having 'a' as values and the same to 'b' and 'c'. I can make use of chunksize in multiprocessing, but here it is not size, its based on key, so how can i achieve this? </p>
  
  <p>Q2: 
  After processing the above chunks, i need the output of all together,order does not matter and then i need to use the whole output for further processing, how can i make my main program wait till all those process complete?</p>
  
  <p>Let me know if further input is required. Thanks.</p>