How to deal with situation where LSTM fails to learn (constantly makes the same incorrect prediction) I am trying to use LSTM neural networks in order to make a song composer. Basically this is based of a text generator (tries to predict the next character after looking at a sequence of characters) but instead of characters, it tried to predict notes.

Structure of the midi file that serves as the input (Y-axis is the pitch or note value while X-axis is time):


And this is the predicted note values:



I set an epoch of 50, but it seems that the LSTM's loss rate does not decrease, most of the time its loss rate does not improve. 



I suspect this is because there is an overwhelming number of a particular note (in this case, note value 65) which makes the LSTM lazy during training phase and predict 65 each and every time.

I feel like this is a common problem among LSTMs and time-series based learning algorithms. How would I solve a problem like this? If what I mentioned is not the problem, then what is the problem and how do I solve that?
Try using a recurrent neural network and use a RProp minus trainer without weight backtracking (aka RProp-, cf. [Igel&amp;Huesken, Neurocomputing 50, 2003]) and without ponderation, ie. all training samples have the same weight.