Is my dataset unlearnable, or is my LSTM model not smart enough?
An LSTM uses the preceding sequence of bitrates to produce a label. If the label can be predicted from those bitrates, then your problem is learnable with an LSTM.
From your comment: The label is the &quot;stage of the video&quot;: Running, walking, etc. The question is if this can be derived from the bitrate? Walking with a busy background might yield a higher bitrate than running. Judging your example by eye, I suspect it cannot. You need to get more features from the video than just the bitrate, I think.