Python NLTK - counting occurrence of word in brown corpora based on returning top results by tag
Question: I'm trying to return the top occurring values from a corpora for specific
tags. I can get the tag and the word themselves to return fine however I can't
get the count to return within the output.

    
    
    import itertools
    import collections
    import nltk 
    from nltk.corpus import brown
    
    words = brown.words()
    
    def findtags(tag_prefix, tagged_text):
    cfd = nltk.ConditionalFreqDist((tag, word) for (word, tag) in tagged_text
                                  if tag.startswith(tag_prefix))
    return dict((tag, cfd[tag].keys()[:5]) for tag in cfd.conditions())
    
    tagdictNNS = findtags('NNS', nltk.corpus.brown.tagged_words())
    

This returns the following fine

    
    
    for tag in sorted(tagdictNNS):
        print tag, tagdictNNS[tag]
    

I have managed to return the count of every NN based word using this:

    
    
    pluralLists = tagdictNNS.values()
    pluralList = list(itertools.chain(*pluralLists)) 
    for s in pluralList:
        sincident = words.count(s)
        print s
        print sincident
    

That returns everything.

Is there a better way of inserting the occurrence into the a dict
`tagdictNN[tag]`?

edit 1:

    
    
    pluralLists = tagdictNNS.values()[:5]
    pluralList = list(itertools.chain(*pluralLists))
    

returns them in size order from the for s loop. still not the right way to do
it though.

edit 2: updated dictionaries so they actually search for NNS plurals.


Answer: I might not understand, but given your tagdictNNS:

    
    
    >>> new = {}
    >>> for k,v in tagdictNNS.items():
            new[k] = len(tagdictNNS[k])
    >>> new
    {'NNS$-TL-HL': 1, 'NNS-HL': 5, 'NNS$-HL': 4, 'NNS-TL': 5, 'NNS-TL-HL': 5, 'NNS+MD': 2,      'NNS$-NC': 1, 'NNS-TL-NC': 1, 'NNS$-TL': 5, 'NNS': 5, 'NNS$': 5, 'NNS-NC': 5}
    

Then you can do something like:

    
    
    >>> sorted(new.items(), key=itemgetter(1), reverse=True)[:2]
    [('NNS-HL', 5), ('NNS-TL', 5)]
    

