Finding top most N common character grams in English
Building up on Caleth's answer:


Beware that:

def ngrams(N, word):
    for i in xrange(len(word) - N):
        yield word[i:i+N]


will not account for the last character:

for g in ngrams(3, "small"):
    print(g)


will output

sma
mal


If it was the purpose of the strict parameter to allow to include/skip the last character (and be able to print the missing all), you can use it that way:

def ngrams(N, words, strict=True):
    last = int(strict)
    for i in xrange(len(word) - N + last):
        yield word[i:i+N]


If, however, you wanted to allow to generate n_grams whose length is lower than N:

def ngrams(N, words, strict=True):
    last = N - 1 if strict else 0
    for i in xrange(len(word) - last):
        yield word[i:i+N]

As per the documentation, FreqDist accepts generators in its constructor. It is thus more memory efficient to turn the n_grams list into a generator:

n_grams = (ngram for ngram in ngrams(N, word) for word in get_words())

Your top comment would be most suited as a docstring for the m_most_common_ngram_chars function. In fact, each function might need its own docstring.
Since almost all your code is functions, you may want to put your last two lines into the module check if __name__ == "__main__":. It will allow you to run your code from the command line as usual and it will do exactly the same thing; but also to import it and call the function with other parameters without running anything at import time.




from nltk import FreqDist
from nltk.corpus import gutenberg


def get_words():
    """generate english words from the whole gutemberg corpus"""
    for fileid in gutenberg.fileids():
        for word in gutenberg.words(fileid):
            yield word

def ngrams(N, word, strict=True):
    """generate a sequence of N-sized substrings of word. 
    if strict is False, also account for P-sized substrings
    at the end of the word where P &lt; N"""

    last = N - 1 if strict else 0
    for i in xrange(len(word) - last):
        yield word[i:i+N]

def m_most_common_ngram_chars(M=5, N=3):
    """gets the top M most common substrings of N characters in English words"""
    f = FreqDist(ngram for ngram in ngrams(N, word) for word in get_words())
    return f.most_common(M)

if __name__ == "__main__":
    # Uses the default values M=5, N=3
    print m_most_common_ngram_chars()