title: Ranking from pairwise comparisons
body: <p>Imagine I have a <strong>very</strong> long list of images and I want to rank them in order of how 'good' people think they are.</p>
  
  <p>I don't want to get users to assign scores to images outright (1 - 10 etc) and order by that, I'd like to try something new.</p>
  
  <p>What I was thinking would be an interesting way of doing it is:</p>
  
  <ul>
  <li>Show a user two random images, they pick the better one</li>
  <li>Collect lots of 'comparisons'</li>
  <li>Use all the comparisons to come up with some ordering</li>
  </ul>
  
  <p>Turns out this is <a href="https://www.google.co.uk/search?q=ranking+from+pairwise+comparisons&amp;oq=ranking+from+pairwise+comparisons&amp;aqs=chrome..69i57j0.4232j0j7&amp;sourceid=chrome&amp;es_sm=119&amp;ie=UTF-8" rel="nofollow">used regularly</a>, for example (using features, not images), this appears to be the way <a href="https://www.uservoice.com/touchpoint-toolkit/" rel="nofollow">Uservoice's Smartvote</a> works.</p>
  
  <p>My question is whether there's a good known way to take this long list of comparisons and build a relative ranking for all the images from them but not to the level of complexity found in the research papers.</p>
  
  <p>I've read a bunch of <a href="http://web.engr.illinois.edu/~swoh/slide_ismp2012.pdf" rel="nofollow">lectures</a> and research papers but I was wondering if there was any sample code out there people might recommend?</p>