title: How do you make predictions with a stateful LSTM?
body: <p>Okay, so I trained a stateful LSTM characterwise on <a href="https://cs.stanford.edu/people/karpathy/char-rnn/shakespear.txt" rel="nofollow noreferrer">https://cs.stanford.edu/people/karpathy/char-rnn/shakespear.txt</a>. It didn't seem to do too bad in terms of accuracy, but know I want to generate my own shakespeare works.</p>
  
  <p>The question is, how do I go about actually generating predictions from it?</p>
  
  <p>In particular, the models batch input shape is (128, 128, 63) and the output shape is (128, 128, 63). (The first number is the batch size, the second number is the length of the prediction input and output, and the third number is the number of distinct characters in the text.)</p>
  
  <p>For example, I would like to:</p>
  
  <ul>
  <li>Generate various predictions starting from empty text</li>
  <li>Generate predictions starting from a small starting text (such as "PYRULEZ:")</li>
  </ul>
  
  <p>This should be possible given how LSTMs work.</p>
  
  <p>Here's a snippet of the code used to generate and fit the model:</p>
  
  <pre><code>model = Sequential()
  model.add(LSTM(dataY.shape[2], batch_input_shape=(128, dataX.shape[1], dataX.shape[2]), return_sequences = True, stateful=True, activation = "softmax"))
  model.summary()
  model.compile(loss='categorical_crossentropy', optimizer='adam', metrics = ['acc'])
  model.fit(dataX, dataY, epochs = 1, batch_size = 128, verbose=1, shuffle = False)
  </code></pre>
  
  <p>Looking at other code samples, it appears I'll need to modify this somehow, but I'm not sure in how specifically.</p>
  
  <p>I can include the whole code sample if that would be helpful. It is self contained.</p>