Obtain a model where each feature vector is past few samples and labels are future few samples? So, I have this data set where each instance is made of past 20 samples of 2 variables. Labels are next five samples. So every instance looks like this:

Instance1_feature = [s1_var1 s2_var1 ... s20_var1 s1_var2 s2_var2 ... s20_var2]
Instance1_label = [s21_var1 s22_var2 ... s25_var5 s21_var2 s22_var2 ... s25_var2]


Instance2_feature = [s2_var1 s3_var1 ... s21_var1 s2_var2 s3_var2... s21_var2]
Instance2_label = [s22_var1 s23_var2 ... s26_var5 s22_var2 s23_var2 ... s26_var2]


All the data are categorical. var1 takes value in {1,2,3,4,5}; var2 takes value in  {6,7,8,9,10}
I am familiar with machine learning. Hence I am looking for a way to develop a model to predict labels for a new instance. I have reasons to believe that by looking at the past samples, it is possible to predict future samples. Its like stock prices.

Any help, links to code samples will be highly appreciated.

EDIT This is a dataset I collected while driving. I want to see if I can predict next 5 seconds of my driving, given I have past 20 seconds of data. That's why I created the feature vectors and labels like this. The values  {1,2,3,4,5} and {6,7,8,9,10} are categorical, as in the variables are split into segments.
This looks like almost an ideal use case for a Recurrent Neural Network (RNN) model. I say almost, as your output variable is 2 separate class vectors, that part is slightly fiddly. 

The main advantage of using a RNN is that the model would be built on the assumption that s1_var1 is of the same type as s2_var1 etc. Whilst with other models you could flatten the time series into a 10 x 20 = 200 long vector, and the output likewise into a 50 long vector, and treat the whole problem as a single step supervised learning, such a model would not include the same assumption.

For your problem, the RNN would have 10 inputs (corresponding to the 5 classes of each of 2 variables) and 10 outputs. You would train it by presenting your input sequence one sample at a time in the correct order 1 to 20, then continue running the network 5 more steps, comparing the outputs against expected. When predicting, you do similar, but then just read off the outputs as predictions, one at a time.

There are a few different Python libraries that support RNN models, and there are different internal choices (such as using LSTM layer versus GRU) which you might want to explore. I am currently learning the Keras framework, which supports some options for RNNs. Probably the example code which learns to predict an exponentially-decaying sine wave would be a good place to start trying to understand the Keras model - although that's a regression, it should give you some insight into how RNNs can be built for this kind of series-based data.



An alternative to RNN here might be a hidden markov model - if you have some insight into the internal state of the system, that could be a better option.