title: How do you implement ranking and sorting in map/reduce?
body: <p>I'm learning the Java map/reduce API in Hadoop and trying to wrap my head around thinking in map/reduce. Here's a sample program i'm writing against apache http server log files, it has two phases (each implemented as a M/R job and then chained together):</p>
  
  <ol>
  <li>Count the number of times each IP address accessed the server</li>
  <li><p>Find the top 5 IP addresses (most requests)</p>
  
  <p>phase 1 seems pretty trivial, it's a simple counting implementation in map/reduce and it emits the something like the following:</p>
  
  <pre><code>192.168.0.2  4
  10.0.0.2  7
  127.0.0.1  3
  ...etc
  </code></pre></li>
  </ol>
  
  <p>This output would feed into the mapper of the second map/reduce job. </p>
  
  <p>Now i'm confused on how to implement the top 5 in a parallel way. Since reducers are sequential in nature, I'm guessing there would only be one reducer that goes against the full list to sort it, right? How do you go about solving step number #2 in a parallel way? </p>