title: How to use keras mask layer correctly?
body: <p>The Keras mask layer can be used to deal with variable-length sequence training of RNNs. When I use them I get lower accuracy with the mask layer than single batch training. I suspect that I'm not using mask layer correctly.</p>
  
  <p>My goal is to train an LSTM to learn how to spell words. The sequences, which are different English words, are encoded with one-hot representation. Below is code of data encoding part: <code>chars</code> are the set of all letters that make up the sequences, <code>mylist</code> is list of the sequences, <code>MAXLEN</code> is the max length of sequences.</p>
  
  <pre><code>char_indices = dict((c, i) for i, c in enumerate(chars))
  indices_char = dict((i, c) for i, c in enumerate(chars))
  
  X = np.zeros((len(mylist), MAXLEN, len(chars)), dtype=np.bool)
  y = np.zeros((len(mylist), MAXLEN, len(chars)), dtype=np.bool)
  
  for i, sentence in enumerate(mylist):
      for t in range(len(sentence)-Data_end):
          X[i, t, char_indices[sentence[t]]] = 1
          y[i, t, char_indices[sentence[t+1]]] = 1
  </code></pre>
  
  <p>My network is defined as:</p>
  
  <pre><code>model = Sequential()
  model.add(Masking(mask_value=0., input_shape=(None, len(chars))))
  model.add(LSTM(2000, return_sequences=True))
  model.add(Dropout(0.2))
  model.add(LSTM(2000, return_sequences=True))
  model.add(Dropout(0.2))
  model.add(TimeDistributed(Dense(len(chars))))
  model.add(Activation('softmax'))
  
  sgd = SGD(lr=lr_init, decay=decay_init, momentum=momentum_init, nesterov=True)
  model.compile(loss='categorical_crossentropy', optimizer=sgd)
  early_stopping = EarlyStopping(patience=2,verbose=1)
  </code></pre>
  
  <p>To train:</p>
  
  <pre><code>model.fit(X, y, callbacks=[early_stopping],batch_size=32, nb_epoch=1)
  </code></pre>
  
  <p>Am I using the mask layer correctly?</p>