CountVectorizer and Out-Of-Vocabulary (OOV) tokens?
There is no inbuilt way in scikit-learn to do this, you need to write some additional code to be able to do this. However you could use the vocabulary_ attribute of CountVectorizer to achieve this.

Cache the current vocabulary
Call fit_transform
Compute the diff with the new vocabulary and the cached vocabulary
Right now I'm using CountVectorizer to extract features.  However, I need to count words not seen during fitting.
During transforming, the default behavior of CountVectorizer is to ignore words that were not observed during fitting.  But I need to keep a count of how many times this happens!
How can I do this?
Thanks!