Training an LSTM to track sine waves
I have added a dense layer to the network:

model = Sequential()

model.add(LSTM(32, return_sequences=True, input_shape=(None, 5)))
model.add(Dropout(0.2))
model.add(LSTM(32, return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(32, return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(32, return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(32, return_sequences=True))
model.add(Dropout(0.2))
model.add(TimeDistributed(Dense(5)))  # &lt;--- This is new :)
model.add(Activation('linear')) 
model.add(Lambda(lambda x: x[:, -50:, :])) # Grab only the last 50 values


This has provided some improvement :



These are two sine waves and the predictions of the last 50 data points of them.
As can be seen, the results have improved, with some of them being okay approximations of the sine waves. Other times... not so much.

Still a lot better than the previous results.

I then tried another approach: Instead of having one (multilayered) LSTM, in which I ignore all but the 50 inputs (in a sense trying to predict from each one of the 50 last values the value located 50 places after it... which might not be an ideal approach) I split the network into 2 LSTMs, with one receiving the input and "injecting" its results via a dense layer into the second LTSM layer which generates the output.

features_num=5 

model.add(LSTM(40, return_sequences=True, input_shape=(None, 5)))
model.add(LSTM(40, return_sequences=False))

model.add(Dense(80))
model.add(Activation('tanh')) 
model.add(RepeatVector(50)) # 50 output points

model.add(LSTM(40, return_sequences=True))
model.add(LSTM(40, return_sequences=True))

model.add(TimeDistributed(Dense(features_num)))
model.add(Activation('linear')) 


Results are, for the most part, better: