title: Lucene: Filtering for documents NOT containing a Term
body: <p>I have an index whose documents have two fields (actually more like 800 fields but the other fields won't concern us here):</p>
  
  <ul>
  <li>The <code>contents</code> field contains the analyzed/tokenized text of the document. The query string is searched for in this field.</li>
  <li>The <code>category</code> field contains the single category identifier of the document. There are about 2500 different categories, and a document may occur in several of them (i.e. a document may have multiple <code>category</code> entries. The results are filtered by this field.</li>
  </ul>
  
  <p>The index contains about 20 mio. documents and is 5 GB in size.</p>
  
  <p>The index is queried with a user-provided query string, plus an optional set of a few categories the user is <em>not</em> interested in. <strong>The question is</strong>: how can I remove those documents matching not only the query string but also the unwanted categories.</p>
  
  <p>I could use a <code>BooleanQuery</code> with a <code>MUST_NOT</code> clause, i.e. something like this:</p>
  
  <pre><code>BooleanQuery q = new BooleanQuery();
  q.add(contentQuery, BooleanClause.MUST);
  for (String unwanted: unwantedCategories) {
      q.add(new TermsQuery(new Term("category", unwanted), BooleanClause.MUST_NOT);
  }
  </code></pre>
  
  <p>Is there a way to do this with Lucene filters? Performance is an issue here, and there will only be a few, recurring, variants of <code>unwantedCategories</code>, so a <code>CachingWrapperFilter</code> would probably help a lot. Also, due to the way the Lucene queries are generated in the existing code base, it is difficult to fit this in, whereas an extra <code>Filter</code> could be introduced easily.</p>
  
  <p>In other words, <strong>How do I create a <code>Filter</code> based on what terms <em>must</em> _not_ occur in a document?</strong></p>