Comparing Language Model of two corpora I know using Conditional Language Model I can learn the probability of a sentence given the corpus I used to train my model. I will then be able to generate meaningful text by sampling from the distribution of sentences.

Now what I want to do is to compare the text generated from the language model of two different corpora on the same topic.

Use case: I want to compare the headline that a right winning vs left winning news outlet would use for a given news content. (my training data would be a large set of headline+news content from both news outlet)
If I understand your question correctly (please comment if I don't), you want to compare the output text from two different language models. Therefore, I don't think you should worry too much about the language models themselves.

If I was you, I'd probably do a simple TFIDF analysis so that you can gain a better understanding of what terms are more prevalent in certain news outlets.

Take a look at scattertext, it allows you to create visualizations like the following: