title: Keras gives nan when training categorical LSTM sequence-to-sequence model
body: <p>I am trying to write a Keras model (using the Tensorflow backend) that uses an LSTM to predict labels for sequences like you would in a part-of-speech labeling task. The model I have written returns <code>nan</code> as a loss for all training epochs and for all label predictions. I suspect I have my model configured incorrectly, but I can't figure out what I'm doing wrong.</p>
  
  <p>The full program is here.</p>
  
  <pre><code>from random import shuffle, sample
  from typing import Tuple, Callable
  
  from numpy import arange, zeros, array, argmax, newaxis
  
  
  def sequence_to_sequence_model(time_steps: int, labels: int, units: int = 16):
      from keras import Sequential
      from keras.layers import LSTM, TimeDistributed, Dense
  
      model = Sequential()
      model.add(LSTM(units=units, input_shape=(time_steps, 1), return_sequences=True))
      model.add(TimeDistributed(Dense(labels)))
      model.compile(loss='categorical_crossentropy', optimizer='adam')
      return model
  
  
  def labeled_sequences(n: int, sequence_sampler: Callable[[], Tuple[array, array]]) -&gt; Tuple[array, array]:
      """
      Create training data for a sequence-to-sequence labeling model.
  
      The features are an array of size samples * time steps * 1.
      The labels are a one-hot encoding of time step labels of size samples * time steps * number of labels.
  
      :param n: number of sequence pairs to generate
      :param sequence_sampler: a function that returns two numeric sequences of equal length
      :return: feature and label sequences
      """
      from keras.utils import to_categorical
  
      xs, ys = sequence_sampler()
      assert len(xs) == len(ys)
      x = zeros((n, len(xs)), int)
      y = zeros((n, len(ys)), int)
      for i in range(n):
          xs, ys = sequence_sampler()
          x[i] = xs
          y[i] = ys
      x = x[:, :, newaxis]
      y = to_categorical(y)
      return x, y
  
  
  def digits_with_repetition_labels() -&gt; Tuple[array, array]:
      """
      Return a random list of 10 digits from 0 to 9. Two of the digits will be repeated. The rest will be unique.
      Along with this list, return a list of 10 labels, where the label is 0 if the corresponding digits is unique and 1
      if it is repeated.
  
      :return: digits and labels
      """
      n = 10
      xs = arange(n)
      ys = zeros(n, int)
      shuffle(xs)
      i, j = sample(range(n), 2)
      xs[j] = xs[i]
      ys[i] = ys[j] = 1
      return xs, ys
  
  
  def main():
      # Train
      x, y = labeled_sequences(1000, digits_with_repetition_labels)
      model = sequence_to_sequence_model(x.shape[1], y.shape[2])
      model.summary()
      model.fit(x, y, epochs=20, verbose=2)
      # Test
      x, y = labeled_sequences(5, digits_with_repetition_labels)
      y_ = model.predict(x, verbose=0)
      x = x[:, :, 0]
      for i in range(x.shape[0]):
          print(' '.join(str(n) for n in x[i]))
          print(' '.join([' ', '*'][int(argmax(n))] for n in y[i]))
          print(y_[i])
  
  
  if __name__ == '__main__':
      main()
  </code></pre>
  
  <p>My feature sequences are arrays of 10 digits from 0 to 9. My corresponding label sequences are arrays of 10 zeros and ones where zero indicates a unique digit and one indicates a repeated digit. (The idea is to create a simple classification task that incorporates long-distance dependencies.)</p>
  
  <p>Training looks like this</p>
  
  <pre><code>Epoch 1/20
   - 1s - loss: nan
  Epoch 2/20
   - 0s - loss: nan
  Epoch 3/20
   - 0s - loss: nan
  </code></pre>
  
  <p>And all the label array predictions look like this</p>
  
  <pre><code>[[nan nan]
   [nan nan]
   [nan nan]
   [nan nan]
   [nan nan]
   [nan nan]
   [nan nan]
   [nan nan]
   [nan nan]
   [nan nan]]
  </code></pre>
  
  <p>So clearly something is wrong.</p>
  
  <p>The features matrix passed to <code>model.fit</code> is of dimensionality <code>samples</code> × <code>time steps</code> × <code>1</code>. The labels matrix is of dimensionality <code>samples</code> × <code>time steps</code> × <code>2</code>, where the 2 comes from a one-hot encoding of the labels 0 and 1.</p>
  
  <p>I'm using a <a href="https://keras.io/layers/wrappers/" rel="nofollow noreferrer">time-distributed dense layer</a> to predict sequences, following the Keras documentation and posts like <a href="https://machinelearningmastery.com/timedistributed-layer-for-long-short-term-memory-networks-in-python/" rel="nofollow noreferrer">this</a> and <a href="https://stackoverflow.com/questions/44611006/timedistributeddense-vs-dense-in-keras-same-number-of-parameters">this</a>. To the best of my knowledge, the model topology defined in <code>sequence_to_sequence_model</code> above is correct. The model summary looks like this</p>
  
  <pre><code>_________________________________________________________________
  Layer (type)                 Output Shape              Param #   
  =================================================================
  lstm_1 (LSTM)                (None, 10, 16)            1152      
  _________________________________________________________________
  time_distributed_1 (TimeDist (None, 10, 2)             34        
  =================================================================
  Total params: 1,186
  Trainable params: 1,186
  Non-trainable params: 0
  _________________________________________________________________
  </code></pre>
  
  <p>Stack Overflow questions like <a href="https://stackoverflow.com/questions/37232782/nan-loss-when-training-regression-network">this</a> make it sound like <code>nan</code> results are an indicator of numeric problems: runaway gradients and whatnot. However, since I am working on a tiny set data and every number that comes back from my model is a <code>nan</code>, I suspect I'm not seeing a numeric problem, but rather a problem with how I have constructed the model.</p>
  
  <p>Does the code above have the right model/data shape for sequence-to-sequence learning? If so, why do I get <code>nan</code>s everywhere?</p>