title: Secondary Sort in Hadoop
body: <p>I am working on a hadoop project and after many visit to various blogs and reading the documentation, I realized I need to use secondry sort feature provided by hadoop framework.</p>
  
  <p>My input format is of the form:</p>
  
  <p><code>DESC(String) Price(Integer) and some other Text</code></p>
  
  <p>I want the values in the reducer to be descending order of the Price.
  Also while comparing DESC I have a method which takes two strings and a Percentage and if similarity between the two strings equals or is greater than the percentage then I should consider them as equal.</p>
  
  <p>The problem is after the Reduce Job is finished I can see some DESC which is similar to the other string and yet they are in different group.</p>
  
  <p>Here is my compareTo method of Composite key</p>
  
  <pre><code>public int compareTo(VendorKey o) {
      int result =-
      result = compare(token, o.token, ":") &gt;= percentage ? 0:1;
      if (result == 0) {
          return pid&gt; o.pid  ?-1: pid &lt; o.pid ?1:0;
      }
      return result;
  }
  </code></pre>
  
  <p>and compare method of Grouping Comparator</p>
  
  <pre><code>public int compare(WritableComparable a, WritableComparable b) {
      VendorKey one = (VendorKey) a;
      VendorKey two = (VendorKey) b;
      int result = ClusterUtil.compare(one.getToken(), two.getToken(), ":") &gt;= one.getPercentage() ? 0 : 1;
      // if (result != 0)
      // return two.getToken().compareTo(one.getToken());
      return result;
  }
  </code></pre>