Text tokenizer - extract words and positions from text I have Set of character delimiters (DELIMITERS), eg . ,  etc. Using this I want to split text and get words with their position in text. 
String.split() works fine if you want only words. The same with StringTokenizer. Wrote some simple method to deal with this, but maybe there is a better way to achieve this result?

public List&lt;String&gt; extractWords(String text){
    List&lt;String&gt; words = new ArrayList&lt;&gt;();
    List&lt;WordPos&gt; positions = new ArrayList&lt;&gt;();
    int wordStart = -1;
    for(int i=0; i &lt; text.length(); i++){
        if(DELIMITERS.contains(text.charAt(i))){
            if(wordStart &gt;=0){ //word just ended
                String word = text.substring(wordStart, i);
                positions.add(new WordPos(wordStart, i));
                words.add(word);
            }
            wordStart = -1;
        }else{ //not delimiter == valid word
            if(wordStart &lt; 0){ //word just started
                wordStart = i;
            }
        }
    }
    return words;
}

// inner static class for words positions
public static class WordPos{
    int start;
    int end;
    public WordPos(int start, int end){
        this.start = start;
        this.end = end;
    }
}
You shouldn't shorten class names. So WordPos should become WordPosition.  



Right now you are creating WordPos items which you add to a List&lt;WordPos&gt; which is lost after returning from the method.  



Instead of using substring() you could use a StringBuilder object, where you append if the current char is contained in DELIMITERS.  

You would need to take care (which you don't do right now) about the case that the text ends without any of your delimiting chars.  

public List&lt;String&gt; extractWords(String text) {
    List&lt;String&gt; words = new ArrayList&lt;&gt;();
    StringBuilder sb = new StringBuilder(text.length());

    for (int i = 0; i &lt; text.length(); i++) {
        char current = text.charAt(i);

        if (DELIMITERS.contains(current)) {
            if (sb.length() &gt; 0) {
                words.add(sb.toString());
                sb.setLength(0);
            }
        } else {
            sb.append(current);
        }
    }
    if (sb.length() &gt; 0) {
        words.add(sb.toString());
    }
    return words;
}