Predicting next number in a sequence - data analysis I am a machine learning newbie and I am working on a project where I'm given a sequence of integers all of which are in the range 0 to 70. My goal is to predict the next integer in the sequence given the previous 5 integers in the same sequence. There isn't much more information on the sequence of integers itself (for example, how was the sequence obtained, etc).

The following are the things I tried.


The first thing that came to mind was to use a LSTM regression model with 5 input time steps and one output (corresponding to the next integer in the sequence - in Keras this would be return_sequences=False). I passed 5 previous integers themselves as the input. This resulted in the model predicting pretty much the average (~30) all the time.
I tried the model in (1) above but with more input time steps (say 100), but still no improvement.
I then tried (1) and (2), but this time using the difference between consecutive integers as the input and trying to predict the difference to the next integer in the sequence. The results with this are still bad.
I then tried a LSTM classification model by one-hot encoding the input and output since I know that all integers in the sequence are in the range 0 to 70. Again, no improvement.
I then tried a seq2seq (encoder-decoder) LSTM model with 5 inputs in the encoder and 5 outputs in the decoder with the correct outputs also being fed into the decoder (teacher forcing). Still the results are bad.


At this point I started doubting whether I can train a model on the given data and whether the given data is just a bunch of random integers.

I looked for statistical tests to determine whether the data is random or not and found out about pandas autocorrelation plot. This is what the plot looks like when plotted on the difference between consecutive integers (looks similar when plotted using the actual integers themselves).



As I understand, since the values are very close to zero, it means that the data is random. Is that right ? 

I also used statsmodels "plot_acf" and the following is the plot I got for the difference between consecutive integers.



I see that there is some negative correlation when the lag is 1. Why doesn't this show up in the plot using pandas' autocorrelation_plot()? 

I tried building a AR (auto regression) model as well, but still the results are bad. 

The histogram of the integers in the sequence also seems to suggest that the integers are random (all values have about the same count except some higher integers).



Am I wasting my time trying to build a machine learning model to predict the next integer in the sequence ?
Interesting problem. The pandas autocorrelation plot suggests that the data is random.

How much do you know about the source?  Is it believable that the sequence is, in fact, random? 

Have you gone through the data and plotted a histogram of the integer appearance counts? Do they appear uniformly or are some more frequent than others?

One thing I think you should try is to reformulate your LSTM model. I don't think this is a regression problem, even though the target is a single integer. It's a classification problem with 70 classes. Try treating it that way and using a categorical crossentropy loss function. In such a scenario, ~1.5% accuracy would represent random guessing. Can the model do better than that?