title: Understanding Tensorflow LSTM models input?
body: <p>I have some trouble understanding LSTM models in TensorFlow.</p>
  
  <p>I use the <a href="http://tflearn.org/" rel="nofollow noreferrer">tflearn</a> as a wrapper, as it does all the initialization and other higher level stuff automatically. For simplicity, let's consider <a href="https://github.com/tflearn/tflearn/blob/master/examples/nlp/lstm.py" rel="nofollow noreferrer">this example program</a>. Until <a href="https://github.com/tflearn/tflearn/blob/master/examples/nlp/lstm.py#L40" rel="nofollow noreferrer">line 42</a>, <code>net = tflearn.input_data([None, 200])</code>, it's pretty clear what happens. You load a dataset into variables and make it of a standard length (in this case, 200). Both the input variables and also the 2 classes are, in this case, converted to one-hot vectors.</p>
  
  <p><em>How does the LSTM take the input</em>? <em>Across how many samples does it predict the output?</em></p>
  
  <p><em>What does <code>net = tflearn.embedding(net, input_dim=20000, output_dim=128)</code> represent?</em></p>
  
  <p>My goal is to <a href="http://jeffdonahue.com/lrcn/" rel="nofollow noreferrer">replicate</a> the activity recognition dataset in the <a href="http://arxiv.org/abs/1411.4389" rel="nofollow noreferrer">paper</a>. For example, I would like to input a 4096 vector as input to the LSTM, and the idea is to take 16 of such vectors, and then produce the classification result. I think the code would look like this, but I don't know how the input to the LSTM should be given.</p>
  
  <pre><code>from __future__ import division, print_function, absolute_import
  
  import tflearn
  from tflearn.data_utils import to_categorical, pad_sequences
  from tflearn.datasets import imdb
  
  train, val = something.load_data()
  trainX, trainY = train #each X sample is a (16,4096) nd float64 
  valX, valY = val #each Y is a one hot vector of 101 classes.
  
  net = tflearn.input_data([None, 16,4096])
  net = tflearn.embedding(net, input_dim=4096, output_dim=256)
  net = tflearn.lstm(net, 256)
  net = tflearn.dropout(net, 0.5)
  net = tflearn.lstm(net, 256)
  net = tflearn.dropout(net, 0.5)
  net = tflearn.fully_connected(net, 101, activation='softmax')
  net = tflearn.regression(net, optimizer='adam',
                           loss='categorical_crossentropy')
  
  model = tflearn.DNN(net, clip_gradients=0., tensorboard_verbose=3)
  model.fit(trainX, trainY, validation_set=(testX, testY), show_metric=True,
            batch_size=128,n_epoch=2,snapshot_epoch=True)
  </code></pre>