title: How to use LSTM for sequence labelling in python?
body: <p>I want to build a classifier that provides labels given a time series of vectors. I have the code for a static LSTM-based classifier, but I don't know how I can incorporate the time information:</p>
  
  <p>Training set:</p>
  
  <pre><code>time   = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10,11,12,13,14,15,16,17,18]
  f1     = [1, 1, 1, 2, 2, 2, 1, 1, 1, 2, 2, 2, 1, 1, 1, 2, 2, 2]
  f2     = [2, 1, 3, 2, 4, 2, 3, 1, 9, 2, 1, 2, 1, 6, 1, 8, 2, 2]
  labels = [a, a, b, b, a, a, b, b, a, a, b, b, a, a, b, b, a, a]
  </code></pre>
  
  <p>Test set:</p>
  
  <pre><code>time   = [1, 2, 3, 4, 5, 6]
  f1     = [2, 2, 2, 1, 1, 1]
  f2     = [2, 1, 2, 1, 6, 1]
  labels = [?, ?, ?, ?, ?, ?]
  </code></pre>
  
  <p>Following <a href="https://stackoverflow.com/questions/25967922/pybrain-time-series-prediction-using-lstm-recurrent-nets">this post</a>, I implemented the following in pybrain:</p>
  
  <pre><code>from pybrain.datasets import SequentialDataSet
  from itertools import cycle
  import matplotlib.pyplot as plt
  from pybrain.tools.shortcuts import buildNetwork
  from pybrain.structure.modules import LSTMLayer
  from pybrain.supervised import RPropMinusTrainer
  from sys import stdout
  
  data = [1,2,3,4,5,6,7]
  
  ds = SequentialDataSet(1, 1)
  for sample, next_sample in zip(data, cycle(data[1:])):
      ds.addSample(sample, next_sample)
  
  print ds
  net = buildNetwork(2, 5, 1, hiddenclass=LSTMLayer, outputbias=False, recurrent=True)
  
  
  trainer = RPropMinusTrainer(net, dataset=ds)
  train_errors = [] # save errors for plotting later
  EPOCHS_PER_CYCLE = 5
  CYCLES = 100
  EPOCHS = EPOCHS_PER_CYCLE * CYCLES
  for i in xrange(CYCLES):
      trainer.trainEpochs(EPOCHS_PER_CYCLE)
      train_errors.append(trainer.testOnData())
      epoch = (i+1) * EPOCHS_PER_CYCLE
      print("\r epoch {}/{}".format(epoch, EPOCHS))
      stdout.flush()
  
  print()
  print("final error =", train_errors[-1])
  
  plt.plot(range(0, EPOCHS, EPOCHS_PER_CYCLE), train_errors)
  plt.xlabel('epoch')
  plt.ylabel('error')
  plt.show()
  
  for sample, target in ds.getSequenceIterator(0):
      print("               sample = %4.1f" % sample)
      print("predicted next sample = %4.1f" % net.activate(sample))
      print("   actual next sample = %4.1f" % target)
      print()
  </code></pre>
  
  <p>This trains a classifier, but I don't know how to incorporate the time information. How can I include the information about the order of the vectors?</p>