title: Python nltk counting word and phrase frequency
body: <p>I am using NLTK and trying to get the word phrase count up to a certain length for a particular document as well as the frequency of each phrase. I tokenize the string to get the data list.</p>
  
  <pre><code>from nltk.util import ngrams
  from nltk.tokenize import sent_tokenize, word_tokenize
  from nltk.collocations import *
  
  
  data = ["this", "is", "not", "a", "test", "this", "is", "real", "not", "a", "test", "this", "is", "this", "is", "real", "not", "a", "test"]
  
  bigrams = ngrams(data, 2)
  
  bigrams_c = {}
  for b in bigrams:
      if b not in bigrams_c:
          bigrams_c[b] = 1
      else:
          bigrams_c[b] += 1
  </code></pre>
  
  <p>the above code gives and output like this:</p>
  
  <pre><code>(('is', 'this'), 1)
  (('test', 'this'), 2)
  (('a', 'test'), 3)
  (('this', 'is'), 4)
  (('is', 'not'), 1)
  (('real', 'not'), 2)
  (('is', 'real'), 2)
  (('not', 'a'), 3)
  </code></pre>
  
  <p>which is partially what I am looking for.</p>
  
  <p>My question is, is there a more convenient way to do this for say up to phrases that are 4 or 5 in length without duplicating this code only to change the count variable?</p>