How to compare the topic coherence between models of different number of topics? If I'm not mistaken, in this paper here http://svn.aksw.org/papers/2015/WSDM_Topic_Evaluation/public.pdf it appears that topics with larger number of topics will inherently have larger coherence measures because of the summation over all the topics. Is there any flaw in dividing this coherence calculation by the number of topics to get an average for a model of 5 topics and then for a model of 10 topics and comparing the two? 

In gensim specifically, I'm using c_v coherence calculation.
After doing some more digging -- it appears that gensim's functions output difference calculations 

This below actually does what I am describing above:

cm = CoherenceModel(model=model, texts=docs, dictionary=dictionary, coherence='c_v')


While this below is actually only the summation across the topics of the model not divided by the total number of topics in the model. 

top_topics = model.top_topics(texts=docs,dictionary=dictionary,coherence='c_v')