Heaps law in Python
<p>I am trying to plot Heaps law for a given text (it shows the growth of vocabulary size in function of the length of the text). That is, for each token i need the length of the text and the vocabulary size up to the given token</p>

<p>I have already tokenised my text, but I am stuck because I don't know how to iterate over all words in the text.</p>

<pre><code>tokens=nltk.wordpunct_tokenize(text)
it=len(tokens)
i=1
for word in tokens:
    print len(tokens), len(set(tokens))
    i=i+1
    if i&gt;it:
        break
</code></pre>

<p>I basically need at each iteration for the text to grow by 1 token.
Thanks for your help!</p>
