Counting the number of unique words in a document with Python
<p>I am  Python newbie trying to understand the answer given <a href="http://stackoverflow.com/questions/914382/how-can-i-count-unique-terms-in-a-plaintext-file-case-insensitively/930185#930185">here</a> to the question of counting unique words in a document. The answer is:</p>

<pre><code>print len(set(w.lower() for w in open('filename.dat').read().split()))
</code></pre>

<blockquote>
  <p>Reads the entire file into memory, splits it into words using
  whitespace, converts each word to lower case, creates a (unique) set
  from the lowercase words, counts them and prints the output</p>
</blockquote>

<p>To try understand that, I am trying to implement it in Python step by step. I can import the text tile using open and read, divide it into individual words using split, and make them all lower case using lower. I can also create a set of the unique words in the list. However, I cannot figure out how to do the last part - count the number of unique words.</p>

<p>I thought I could finish by iterating through the items in the set of unique words and counting them in the original lower-case list, but I find that that the set construct is not indexable. </p>

<p>So I guess I am trying to do something that in natural language is like, for all the items in the set, tell me how many times they occur in the lower case list. But I cannot quite figure out how to do that, and I suspect some underlying misunderstanding of Python is holding me back.</p>

<ul>
<li>EDIT - </li>
</ul>

<p>Guys thanks for the answers. I have just realised I did not explain myself correctly - I wanted to find not only the total number of unique words (which I understand is the length of the set) but also the number of times each individual word was used, e.g. 'the' was used 14 times, 'and' was used 9 times, 'it' was used 20 times and so on. Apologies for the confusion.</p>
