title: How to interpret weights in a LSTM layer in Keras
body: <p>I'm currently training a recurrent neural network for weather forecasting, using a LSTM layer. The network itself is pretty simple and looks roughly like this:</p>
  
  <pre><code>model = Sequential()  
  model.add(LSTM(hidden_neurons, input_shape=(time_steps, feature_count), return_sequences=False))  
  model.add(Dense(feature_count))  
  model.add(Activation("linear"))  
  </code></pre>
  
  <p>The weights of the LSTM layer do have the following shapes:</p>
  
  <pre><code>for weight in model.get_weights(): # weights from Dense layer omitted
      print(weight.shape)
  
  &gt; (feature_count, hidden_neurons)
  &gt; (hidden_neurons, hidden_neurons)
  &gt; (hidden_neurons,)
  &gt; (feature_count, hidden_neurons)
  &gt; (hidden_neurons, hidden_neurons)
  &gt; (hidden_neurons,)
  &gt; (feature_count, hidden_neurons)
  &gt; (hidden_neurons, hidden_neurons)
  &gt; (hidden_neurons,)
  &gt; (feature_count, hidden_neurons)
  &gt; (hidden_neurons, hidden_neurons)
  &gt; (hidden_neurons,)
  </code></pre>
  
  <p>In short, it looks like there are four "elements" in this LSTM layer. I'm wondering now how to interpret them:</p>
  
  <ul>
  <li><p>Where is the <code>time_steps</code> parameter in this representation? How does it influence the weights?</p></li>
  <li><p>I've read that a LSTM consists of several blocks, like an input and a forget gate. If those are represented in these weight matrices, which matrix belongs to which gate?</p></li>
  <li><p>Is there any way to see what the network has learned? For example, how much does it take from the last time step (<code>t-1</code> if we want to forecast <code>t</code>) and how much from <code>t-2</code> etc? It would be interesting to know if we could read from the weights that the input <code>t-5</code> is completely irrelevant, for example.</p></li>
  </ul>
  
  <p>Clarifications and hints would be greatly appreciated.</p>