title: Error when stripping punctuation from corpus
body: <p>Thank you in advance for your help. I'm trying to write a script that will look at a corpus, find all trigrams and print those along with their relative frequencies into a csv file. I have gotten pretty far but keep running into one problem. It thinks conjunctions are two words because of the apostrophe so it splits <code>doesn't</code> into <code>doesn t</code>, which messes up the trigram count. I am trying to solve that problem by removing all punctuation from the raw variable, which I believe is just one long string that contains all of the text from my corpus with this line:</p>
  
  <pre><code>    raw = raw.translate(None, string.punctuation)
  </code></pre>
  
  <p>But that gives me an error that says: <code>NameError: name 'string' is not defined</code></p>
  
  <p>But I didn't think string had to be defined when used like that? Does that mean raw is not a string? How can solve this? </p>
  
  <pre><code>#this imports the text files in the folder into corpus called speeches
  corpus_root = '/Users/root'
  speeches = PlaintextCorpusReader(corpus_root, '.*\.txt') 
  print "Finished importing corpus"
  tokenizer = RegexpTokenizer(r'\w+')
  raw = speeches.raw().lower()
  raw = raw.translate(None, string.punctuation)
  finalwords = raw.encode['ascii','xmlcharrefreplace']
  tokens = tokenizer.tokenize(finalwords)
  tgs = nltk.trigrams(tokens)
  fdist = nltk.FreqDist(tgs)
  minscore = 40
  numwords = len(finalwords)
  print "Words in corpus:" 
  print numwords
  c = csv.writer(open("TPNngrams.csv", "wb"))
  for k,v in fdist.items():
      if v &gt; minscore:
          rf = Decimal(v)/Decimal(numwords)
          firstword, secondword, thirdword = k
          trigram = firstword + " " + secondword + " " + thirdword
          results = trigram,v,rf
          c.writerow(results)
          print firstword, secondword, thirdword, v, rf
  
  print "All done."
  </code></pre>