Creating an inverted index in Python Here is the code I have written to create an inverted index dictionary for a set of documents:

inv_indx = {i:[] for i in corpus_dict}
for word in corpus_dict:
    for i in range(len(docs)):
        if word in docs[i]:
            inv_indx[word].append(i)


docs is a list of sets of the words in various documents:

[{'once','upon','a','time',...},{'lorum','ipsum','time'...},...]


corpus_dict is a set of all the words that appear in any of the documents:

{'once','upon','a','time','lorum','ipsum',...}


inv_index becomes a dictionary with each word in the corpus_dict as a key for a list of the document ids that contain that word:

{'once':[0],'time':[0,1],...}


The problem is this becomes very slow if the number of documents gets too big. How can I make this code more efficient?
Suggestions


You check all words in all documents. Try iterate only over docs (without unnecessary checks)
instead of create empty inv_indx = {i:[] for i in corpus_dict} you can use defaultdict


Code

from collections import defaultdict

inv_indx = defaultdict(list)
for idx, text in enumerate(docs):
    for word in text:
        inv_indx[word].append(idx)