if you are going to explore the topics yourself or build visualization toolyou should probably try few values and see which gives you the most useful or most appealing results howeverthere are few methods that will automatically determine the number of topics for you depending on the dataset one popular model is called the hierarchical dirichlet process againthe full mathematical model behind it is complex and beyond the scope of this bookbut the fable we can tell is that instead of having the topics be fixed priori and our task being to reverse engineer the data to get them backthe topics themselves were generated along with the data whenever the writer was going to start new documenthe had the option of using the topics that already existed or creating completely new one this means that the more documents we havethe more topics we will end up with this is one of those statements that is unintuitive at firstbut makes perfect sense upon reflection we are learning topicsand the more examples we havethe more we can break them up if we only have few examples of news articlesthen sports will be topic howeveras we have morewe start to break it up into the individual modalities such as hockeysoccerand so on as we have even more datawe can start to tell nuances apart articles about individual teams and even individual players the same is true for people in group of many different backgroundswith few "computer people"you might put them togetherin slightly larger groupyou would have separate gatherings for programmers and systems managers in the real worldwe even have different gatherings for python and ruby programmers one of the methods for automatically determining the number of topics is called the hierarchical dirichlet process (hdp)and it is available in gensim using it is trivial taking the previous code for ldawe just need to replace the call to gensim models ldamodel ldamodel with call to the hdpmodel constructor as followshdp gensim models hdpmodel hdpmodel(mmid wordthat' it (except it takes bit longer to compute--there are no free lunchesnowwe can use this model as much as we used the lda modelexcept that we did not need to specify the number of topics summary in this we discussed more advanced form of grouping documentswhich is more flexible than simple clustering as we allow each document to be present in more than one group we explored the basic lda model using new packagegensimbut were able to integrate it easily into the standard python scientific ecosystem