title: tf-idf and previously unseen terms
body: <p><a href="http://en.wikipedia.org/wiki/Tf-idf" rel="noreferrer">TF-IDF (term frequency - inverse document frequency)</a> is a staple of information retrieval.  It's not a proper model though, and it seems to break down when new terms are introduced into the corpus.  How do people handle it when queries or new documents have new terms, especially if they are high frequency.  Under traditional cosine matching, those would have no impact on the total match.  </p>