<p>I was going to Ask a Question earlier today when I was presented to a surprising functionality in Stackoverflow. When I wrote my question title stackoverflow suggested me several related questions and I found out that there was already two similar questions. That was stunning! </p>

<p>Then I started thinking how I would implement such function. How I would order questions by relatedness:</p>

<ol>
<li>Question that have higher number of
words matchs with the new question</li>
<li>If the number of matchs are the
same, the order of words is considered</li>
<li>Words that appears in the title has
higher relevancy</li>
</ol>

<p>That would be a simple workflow or a complex score algortithm?
Some stemming to increase the recall, maybe?
Is there some library the implements this function?
What other aspects would you consider?
Maybe Jeff could answer himself! How did you implemented this in Stackoverflow? :)</p>

[{'answer_id': 42496, 'author': 'Paul Tomblin', 'author_id': 3333, 'author_profile': 'https://Stackoverflow.com/users/3333', 'pm_score': -1, 'selected': False, 'text': "<p>Isn't StackOverflow going to be open sourced at some point?  If so, you can always find out how they did it there.</p>\n\n<p>Update: It appears that they say they <em>might</em> open source it.  I hope they do.</p>\n"}, {'answer_id': 42532, 'author': 'Pascal', 'author_id': 1311, 'author_profile': 'https://Stackoverflow.com/users/1311', 'pm_score': 3, 'selected': True, 'text': "<p>One such way to implement such an algorithm would involve ranking the questions as per a heuristic function which assigns a 'relevance' weight factor using the following steps:</p>\n\n<ol>\n<li>Apply a noise filter to the 'New' question to remove words that are common across a large number of objects such as: 'the', 'and', 'or', etc.</li>\n<li>Get the number of words contained in the 'New' question which match the words the set of questions already posted on the website. [A]</li>\n<li>Get the number of tag matches between the words in the 'New' question and the available. [B]</li>\n<li>Compute the 'relevance weight' based on [A] and [B] as 'x[A] + y[B]', where x and y are weight multipliers (Assign a higher weight multiplier to [B] as tagging is more relevant than simple word search)</li>\n<li>Get the top 5 questions which have the highest 'relevance weight'.</li>\n</ol>\n\n<p>The heuristic might require tweaking to get optimal results, but it should work.</p>\n"}, {'answer_id': 42598, 'author': 'Pascal', 'author_id': 1311, 'author_profile': 'https://Stackoverflow.com/users/1311', 'pm_score': 0, 'selected': False, 'text': '<p><a href="https://stackoverflow.com/questions/42489/how-to-implement-a-related-degree-measure-algorithm#42545">@marcio</a></p>\n\n<p>Sorry, I am not aware of any direct API reference that I could suggest here and I have never worked with Lucene.</p>\n\n<p>However, I am aware that Google Desktop uses a Query API to rank and suggest the relevant search results. More information on the API can be found <a href="http://code.google.com/apis/desktop/docs/queryapi.html" rel="nofollow noreferrer">here</a>.</p>\n\n<p>Perhaps others could chime in and guide you. </p>\n'}, {'answer_id': 43786, 'author': 'robaker', 'author_id': 190, 'author_profile': 'https://Stackoverflow.com/users/190', 'pm_score': 1, 'selected': False, 'text': '<p>Your question seems similar to <a href="https://stackoverflow.com/questions/41424/how-do-you-implement-a-did-you-mean">this one</a>, which has some additional answers.</p>\n'}]
['https://Stackoverflow.com/questions/42489', 'https://Stackoverflow.com', 'https://Stackoverflow.com/users/4213/']