title: How do I create a variable-length input LSTM in Keras?
body: <p>I am trying to do some vanilla pattern recognition with an LSTM using Keras to predict the next element in a sequence.</p>
  
  <p>My data look like this:</p>
  
  <p><a href="https://i.stack.imgur.com/NnEvI.png" rel="noreferrer"><img src="https://i.stack.imgur.com/NnEvI.png" alt="My data"></a></p>
  
  <p>where the label of the training sequence is the last element in the list: <code>X_train['Sequence'][n][-1]</code>.</p>
  
  <p>Because my <code>Sequence</code> column can have a variable number of elements in the sequence, I believe an RNN to be the best model to use. Below is my attempt to build an LSTM in Keras:</p>
  
  <pre><code># Build the model
  
  # A few arbitrary constants...
  max_features = 20000
  out_size = 128
  
  # The max length should be the length of the longest sequence (minus one to account for the label)
  max_length = X_train['Sequence'].apply(len).max() - 1
  
  # Normal LSTM model construction with sigmoid activation
  model = Sequential()
  model.add(Embedding(max_features, out_size, input_length=max_length, dropout=0.2))
  model.add(LSTM(128, dropout_W=0.2, dropout_U=0.2))
  model.add(Dense(1))
  model.add(Activation('sigmoid'))
  
  # try using different optimizers and different optimizer configs
  model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
  </code></pre>
  
  <p>And here's how I attempt to train my model:</p>
  
  <pre><code># Train the model
  for seq in X_train['Sequence']:
      print("Length of training is {0}".format(len(seq[:-1])))
      print("Training set is {0}".format(seq[:-1]))
      model.fit(np.array([seq[:-1]]), [seq[-1]])
  </code></pre>
  
  <p>My output is this:</p>
  
  <pre><code>Length of training is 13
  Training set is [1, 3, 13, 87, 1053, 28576, 2141733, 508147108, 402135275365, 1073376057490373, 9700385489355970183, 298434346895322960005291, 31479360095907908092817694945]
  </code></pre>
  
  <p>However, I get the following error:</p>
  
  <pre><code>Exception: Error when checking model input: expected embedding_input_1 to have shape (None, 347) but got array with shape (1, 13)
  </code></pre>
  
  <p>I believe my training step is correctly setup, so my model construction must be wrong. Note that 347 is <code>max_length</code>.</p>
  
  <p>How can I correctly build a variable-length input LSTM in Keras? I'd prefer not to pad the data. Not sure if it's relevant, but I'm using the Theano backend.</p>