An easy question perhaps?Taking a (fictional) concrete example, let's say I have two ranking methods for HTML documents: PageRank and HITS. I derive an ordered list over the same set of documents according to their rank values (there can be ties) for both ranking methods. Now, I want to quantify how much the two orderings correlate. I'm not interested in the absolute values of the ranking measures, but I'm looking for measures of correlation between the orderings.I can think of some intuitive correlation measures one could apply like average distance or such: ${\displaystyle\frac{\sum_{doc_i\in Docs}\mathrm{abs}(pos_{rank1}(doc_i) - pos_{rank2}(doc_i))}{|Docs|^2}}$but I'm wondering if anyone knows of any standard techniques I could check out? In particular, I would prefer a symmetric measure which is easy to intuit and whose results are tangible/portable. I've a number of different orderings which I'll be comparing pair-wise, but the set of $Docs$ remains the same.Many thanks in advance! (having difficulty tagging... please feel free to edit tags)
Measures of correlation between two orderings
co.combinatorics;st.statistics;order theory
The standard approach to computing the correlation between two total orders is Kendall's $\tau$, defined as the difference between the number of agreeing pairs and disagreeing pairs (in rank) divided by $\binom{n}{2}$. If you only have partial orders, then things get a little trickier. 