title: Best practices for seaching for alternate forms of a word with Lucene
body: <p>I have a site which is searchable using Lucene.  I've noticed from logs that users sometimes don't find what they're looking for because they enter a singular term, but only the plural version of that term is used on the site.  I would like the search to find uses of other forms of a word as well.  This is a problem that I'm sure has been solved many times over, so what are the best practices for this?</p>
  
  <p><strong>Please note: this site only has English content</strong>.</p>
  
  <p>Some approaches I've thought of:</p>
  
  <ol>
  <li>Look up the word in some kind of thesaurus file to determine alternate forms of a given word.
  
  <ul>
  <li>Some examples:
  
  <ul>
  <li>Searches for "car", also add "cars" to the query.</li>
  <li>Searches for "carry", also add "carries" and "carried" to the query.</li>
  <li>Searches for "small", also add "smaller" and "smallest" to the query.</li>
  <li>Searches for "can", also add "can't", "cannot", "cans", and "canned" to the query.</li>
  <li>And it should work in reverse (i.e. search for "carries" should add "carry" and "carried").</li>
  </ul></li>
  <li>Drawbacks:
  
  <ul>
  <li>Doesn't work for many new technical words unless the dictionary/thesaurus is updated frequently.</li>
  <li>I'm not sure about the performance of searching the thesaurus file.</li>
  </ul></li>
  </ul></li>
  <li>Generate the alternate forms algorithmically, based on some heuristics.
  
  <ul>
  <li>Some examples:
  
  <ul>
  <li>If the word ends in "s" or "es" or "ed" or "er" or "est", drop the suffix</li>
  <li>If the word ends in "ies" or "ied" or "ier" or "iest", convert to "y"</li>
  <li>If the word ends in "y", convert to "ies", "ied", "ier", and "iest"</li>
  <li>Try adding "s", "es", "er" and "est" to the word.</li>
  </ul></li>
  <li>Drawbacks:
  
  <ul>
  <li>Generates lots of non-words for most inputs.</li>
  <li>Feels like a hack.</li>
  <li>Looks like something you'd find on TheDailyWTF.com. :)</li>
  </ul></li>
  </ul></li>
  <li>Something much more sophisticated?</li>
  </ol>
  
  <p>I'm thinking of doing some kind of combination of the first two approaches, but I'm not sure where to find a thesaurus file (or what it's called, as "thesaurus" isn't quite right, but neither is "dictionary").</p>