How to check if a word is more common in its plural form rather than in it's singular form in an array of words (with Python/NLTK)?
Question: I'm trying to do the NLTK exercices but I can't do this one. "Which nouns are
more common in their plural form, rather than their singular form? (Only
consider regular plurals, formed with the -s suffix.)". I spent a day thinking
about this and trying things but I just can't get it. Thank you.


Answer: Take a corpus, do a count_:

    
    
    >>> from collections import Counter
    >>> from nltk.corpus import brown
    >>> texts = brown.words()[:10000]
    >>> word_counts = Counter(texts)
    >>> word_counts['dollar']
    5
    >>> word_counts['dollars']
    15
    

But do note that sometimes it's unclear when you use only surface strings in
counting, e.g.

    
    
    >>> texts = brown.words()[:10000]
    >>> word_counts = Counter(texts)
    >>> word_counts['hits']
    14
    >>> word_counts['hit']
    34
    >>> word_counts['needs']
    14
    >>> word_counts['need']
    30
    

POS sensitive counts (see types vs tokens):

    
    
    >>> texts = brown.tagged_words()[:10000]
    >>> word_counts = Counter(texts)
    >>> word_counts[('need', 'NN')]
    6
    >>> word_counts[('needs', 'NNS')]
    3
    >>> word_counts[('hit', 'NN')]
    0
    >>> word_counts[('hits', 'NNS')]
    0
    

Let's reverse engineer a little, `brown` corpus is nice and it's tokenized and
tagged in NLTK but if you want to use your own corpus, then you have to
consider the follow:

  * Which corpus to use? How to tokenize? How to POS-tag?
  * What are you counting? Types or tokens?
  * How to handle POS ambiguity? How to differentiate Nouns from non-Nouns?

Finally, consider this:

  * Is there really a way to find out whether plural or singular is more common for a word in language? Or will it always be relative to the corpus you chose to analyze?
  * Are there cases where plural or singular don't exists for certain nouns? (Most probably the answer is yes). 

