Evaluating Language Model on specific topic
What you can do is to compare against a validation set of the same domain. First, you use your LM to generate many sentences, and, for each sentence, you compute the BLEU score against the whole validation set. This python script may be useful for that.
However, you should take into account that it is possible that your model generates very similar sentences always. For that reason, people normally evaluate not only quality but also diversity. For this, you can compute the &quot;self BLEU&quot; of your generated sentences, that is, you compute the BLEU of each generated sentence against the rest of the generated sentences. You can find a script for that here.
You should also take into account that you can trade quality for diversity and vice versa, by setting different values of the output softmax temperature. For this, you may want to evaluate your model at different values of the temperature, to understand different generation regimes, plotting them as a curve, like this:

You can find more about this kind of evaluation in this article, where I took the figure from.
You can find alternatives to the quality-diversity evaluation here.