title: Time series prediction with keras and multiple sequences
body: <p>I understand the <a href="https://github.com/fchollet/keras/blob/master/examples/stateful_lstm.py" rel="nofollow">stateful LSTM prediction example</a> in Keras on a single sequence. That example has one sequence of 50k observations. </p>
  
  <p>My questions:</p>
  
  <ul>
  <li>What if you want to train multiple sequences of 50k observations? Say one that starts/ends at different values and has a slightly different behavior?</li>
  <li>How to modify the example to increase the prediction time step?</li>
  <li>Are LSTMs even any good for that sort of thing?</li>
  </ul>
  
  <p>Fully replicable example with 3 mean-reverting time series and predicting 20 steps out.</p>
  
  <pre><code># generate random data
  import statsmodels.api as sm
  import numpy as np
  import pandas as pd
  
  cfg_t_total = 25000
  cfg_t_step = 20
  cfg_batch_size = 100
  
  np.random.seed(12345)
  arparams = np.array([.75, -.25])
  maparams = np.array([.65, .35])
  ar = np.r_[1, -arparams] # add zero-lag and negate
  ma = np.r_[1, maparams] # add zero-lag
  y0 = sm.tsa.arma_generate_sample(ar, ma, cfg_t_total)
  y1 = sm.tsa.arma_generate_sample(ar, ma, cfg_t_total)
  y2 = sm.tsa.arma_generate_sample(ar, ma, cfg_t_total)
  
  df=pd.DataFrame({'a':y0,'b':y1,'c':y2})
  
  df.head(100).plot()
  
  df.head(5)
  
  # create training data format
  X = df.unstack()
  y = X.groupby(level=0).shift(-cfg_t_step)
  
  idx_keep = ~(y.isnull())
  X = X.ix[idx_keep]
  y = y.ix[idx_keep]
  
  from keras.models import Sequential
  from keras.layers import Dense, LSTM
  
  # LSTM taken from https://github.com/fchollet/keras/blob/master/examples/stateful_lstm.py
  # how to do this...?!
  print('Creating Model')
  model = Sequential()
  model.add(LSTM(50,
                 batch_input_shape=(cfg_batch_size, cfg_t_step, 1),
                 return_sequences=True,
                 stateful=True))
  model.add(LSTM(50,
                 batch_input_shape=(cfg_batch_size, cfg_t_step, 1),
                 return_sequences=False,
                 stateful=True))
  model.add(Dense(1))
  model.compile(loss='mse', optimizer='rmsprop')
  
  model.fit(X, y, batch_size=cfg_batch_size, verbose=2, validation_split=0.25, nb_epoch=1, shuffle=False)
  </code></pre>