title: Text tokenizer - extract words and positions from text
body: <p>I have <code>Set</code> of <code>character</code> delimiters (<code>DELIMITERS</code>), eg <code>.</code> <code>,</code> <code></code> etc. Using this I want to split text and get words with their position in text. 
  <code>String.split()</code> works fine if you want only words. The same with <code>StringTokenizer</code>. Wrote some simple method to deal with this, but maybe there is a better way to achieve this result?</p>
  
  <pre><code>public List&lt;String&gt; extractWords(String text){
      List&lt;String&gt; words = new ArrayList&lt;&gt;();
      List&lt;WordPos&gt; positions = new ArrayList&lt;&gt;();
      int wordStart = -1;
      for(int i=0; i &lt; text.length(); i++){
          if(DELIMITERS.contains(text.charAt(i))){
              if(wordStart &gt;=0){ //word just ended
                  String word = text.substring(wordStart, i);
                  positions.add(new WordPos(wordStart, i));
                  words.add(word);
              }
              wordStart = -1;
          }else{ //not delimiter == valid word
              if(wordStart &lt; 0){ //word just started
                  wordStart = i;
              }
          }
      }
      return words;
  }
  
  // inner static class for words positions
  public static class WordPos{
      int start;
      int end;
      public WordPos(int start, int end){
          this.start = start;
          this.end = end;
      }
  }
  </code></pre>