title: Hadoop 2.2 - Combine Reducer Output
body: <p>My Map/Reduce job outputs lines of the form</p>
  
  <pre><code>Key1 5
  Key2 8
  Key3 4
  Key1 7
  Key3 3
  </code></pre>
  
  <p>And I would like to aggregate these results to get the sum of the values by key.
  So something like:</p>
  
  <pre><code>Key1 12 (5 + 7)
  Key2 8
  Key3 7 (4 + 3)
  </code></pre>
  
  <p>The naive way would be to simply chain another Map/Reduce job behind the first one but it is inefficient because you have twice more I/O than necessary, the overhead, etc. So I would like to avoid this solution.</p>
  
  <p>The best way to do it would probably be something like <code>map -&gt; reduce1 -&gt; reduce2</code> where the output of reduce1 is the input of reduce2. Unfortunately, it appears to be impossible to do (see <a href="https://stackoverflow.com/questions/16870675/chaining-multi-reducers-in-a-hadoop-mapreduce-job">Chaining Multi-Reducers in a Hadoop MapReduce job</a> for example).
  I have also looked at <a href="https://hadoop.apache.org/docs/r2.2.0/api/org/apache/hadoop/mapreduce/lib/chain/ChainReducer.html" rel="nofollow noreferrer">ChainReducer</a> but it doesn't help either since it doesn't allow several reduce steps.</p>
  
  <p>So how would you go about it? Is there something new in Hadoop 2.X that I could use?</p>
  
  <p>Thank you.</p>