Where can I find pre-trained language models in English and German?
Of course now there has been a huge development:
Huggingface published pytorch-transformers, a library for the so successful Transformer models (BERT and its variants, GPT-2, XLNet, etc.), including many pretrained (mostly English or multilingual) models (docs here). It also includes one German BERT model. SpaCy offers a convenient wrapper (blog post).

Update: Now, Salesforce published the English model CTRL, which allows for use of "control codes" that influence the style, genre and content of the generated text.

For completeness, here is the old, now less relevant version of my answer:



Since I posed the question, I found this pretrained German language model:
https://lernapparat.de/german-lm/

It is an instance of a 3-layer "averaged stochastic descent weight-dropped" LSTM which was implemented based on an implementation by Salesforce.