keras CNN lstm add model depth
CTry to change 

model.add(LSTM(50, activation='relu'))


to 

model.add(LSTM(50, activation='relu', return_sequences=True))


as explained in the documentation ( https://keras.io/layers/recurrent/#lstm ) return_sequences allow the LSTM to return the complete sequence of vectors of the LSTM rather than a single vector.