title: Having trouble rereading a Lucene TokenStream
body: <p>I am using Lucene 4.6, and am apparently unclear on how to reuse a TokenStream, because I get the exception:</p>
  
  <pre><code>java.lang.IllegalStateException: TokenStream contract violation: reset()/close() call missing, reset() called multiple times, or subclass does not call super.reset(). Please see Javadocs of TokenStream class for more information about the correct consuming workflow.
  </code></pre>
  
  <p>at the start of the second pass. I've read the Javadoc, but I'm still missing something. Here is a simple example that throws the above exception:</p>
  
  <pre><code>@Test
  public void list() throws Exception {
    String text = "here are some words";
    TokenStream ts = new StandardTokenizer(Version.LUCENE_46, new StringReader(text));
    listTokens(ts);
    listTokens(ts);
  }
  
  public static void listTokens(TokenStream ts) throws Exception {
    CharTermAttribute termAtt = ts.addAttribute(CharTermAttribute.class);
    try {
      ts.reset();
      while (ts.incrementToken()) {
        System.out.println("token text: " + termAtt.toString());
      }
      ts.end();
    }
    finally {
      ts.close();
    }
  }
  </code></pre>
  
  <p>I've tried not calling <code>TokenStream.end()</code> or <code>TokenStream.close()</code> thinking maybe they should only be called at the very end, but I get the same exception.</p>
  
  <p>Can anyone offer a suggestion?</p>