title: How to find the lemmas and frequency count of each word in list of sentences in a list?
body: <p>I want to find out the lemmas using WordNet Lemmatizer and also I need to compute each word frequency. </p>
  
  <p>I am getting the following error.</p>
  
  <p>The trace is as follows:</p>
  
  <p><a href="https://i.stack.imgur.com/VgvFw.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/VgvFw.png" alt="enter image description here"></a></p>
  
  <blockquote>
    <p>TypeError: unhashable type: 'list'</p>
  </blockquote>
  
  <p>Note: The corpus is available on the <code>nltk</code> package itself.  </p>
  
  <p>What I have tried so far is as follows:</p>
  
  <pre><code>import nltk, re
  import string
  from collections import Counter
  from string import punctuation
  from nltk.tokenize import TweetTokenizer, sent_tokenize, word_tokenize
  from nltk.corpus import gutenberg, stopwords
  from nltk.stem import WordNetLemmatizer
  
  def remove_punctuation(from_text):
      table = str.maketrans('', '', string.punctuation)
      stripped = [w.translate(table) for w in from_text]
      return stripped
  
  def preprocessing():
      raw_data = (gutenberg.raw('shakespeare-hamlet.txt'))
      tokens_sentences = sent_tokenize(raw_data)
      tokens = [[word.lower() for word in line.split()] for line in tokens_sentences]
      print(len(tokens))
      global stripped_tokens
      stripped_tokens = [remove_punctuation(i) for i in tokens]
      sw = (stopwords.words('english'))
      filter_set = [[token for token in sentence if (token.lower() not in sw and token.isalnum())] for sentence in stripped_tokens]
      lemma= WordNetLemmatizer()
      global lem
      lem = []
      for w in filter_set:
          lem.append(lemma.lemmatize(w))
  
  preprocessing()
  </code></pre>
  
  <p>Please help me in resolving the issue.</p>