LSTM language model not working
You can follow the below steps :


LSTMs are slower in terms of convergence. They take much time to train amd thereby give better results. Try training the network for a more number of epochs like 50 or 65.
Use smaller batch size. Try using a RMSProp optimizer. It takes a long time to converge but gives excellent results in case of recurrent nets.
Also, try word  level predictions. Means, train your LSTM in such a way that it can predict words and not characters.
Use dropout layers in between two LSTM layers. Use a rate like 0.2 or 0.35.


As a working example, you can refer this 
project which generates poem lines using LSTM in tensorflow.