title: Lucene.NET TokenStream.Next method disappeared
body: <p>I have to update a project using Lucene.NET. It first time I meet this library, and I should update the references to a new version. So I did with Lucene references via a NuGet.</p>
  
  <p>But actually I should in a way update some methods that disappeared in the new versions.</p>
  
  <pre><code>public abstract class AnalyzerView
  {
      public abstract string Name { get; }
  
      public virtual string GetView(TokenStream tokenStream,out int numberOfTokens)
      {
          StringBuilder sb = new StringBuilder();
  
          Token token = tokenStream.Next();
  
          numberOfTokens = 0;
  
          while (token != null)
          {
              numberOfTokens++;
              sb.Append(GetTokenView(token));
              token = tokenStream.Next();
          }
  
          return sb.ToString();
      }
  
      protected abstract string GetTokenView(Token token);
  }
  </code></pre>
  
  <p>The documentation is not very clear for me. As I understood I should use something like </p>
  
  <pre><code>while (tokenStream.IncrementToken()) { ... }
  </code></pre>
  
  <p>but how to obtain that following token is a mystery for me. </p>
  
  <p>I saw <a href="https://stackoverflow.com/q/2638200/185593">similar question</a> on StackOverflow, but my problem is that I need to obtain a <code>Token</code> rather thatn a <code>string</code>, because I have some code that uses tokens, like this one:</p>
  
  <pre><code>public class TermWithOffsetsView : AnalyzerView {
      public override string Name {
          get { return "Terms With Offsets"; }
      }
  
      protected override string GetTokenView(Token token) {
          return token.TermText() + 
              "   Start: " + token.StartOffset().ToString().PadLeft(5) + 
              "  End: " + token.EndOffset().ToString().PadLeft(5) + 
              "\r\n";
      }
  }
  </code></pre>