ValueError: cannot compute LDA over an empty collection (no terms)
Finally figured it out. The issue with small documents is that if you try to filter the extremes from dictionary, you might end up with empty lists in corpus.corpus = [dictionary.doc2bow(text)].
So the values of parameters in dictionary.filter_extremes(no_below=2, no_above=0.1) needs to be selected accordingly and carefully before corpus = [dictionary.doc2bow(text)]
I just removed the filter extremes and lda model runs fine now. Though I will change the parameter values in filter extreme and use it later.
Getting this error in python when trying to compute lda for a smaller size of corpus but works fine in other cases.
The size of corpus is 15 and I tried setting the number of topic to 5 then reduced it to 2 but it still gives the same error : ValueError: cannot compute LDA over an empty collection (no terms)
getting error at this line :     lda = models.LdaModel(corpus, num_topics=topic_number, id2word=dictionary, passes=passes)
where corpus is corpus = [dictionary.doc2bow(text) for a, id, text, s_date, e_date, qd, qd_perc in texts]
Why is it giving no terms?