title: What's the difference between "hidden" and "output" in PyTorch LSTM?
body: <p>I'm having trouble understanding the documentation for PyTorch's LSTM module (and also RNN and GRU, which are similar). Regarding the outputs, it says:</p>
  
  <blockquote>
    <p>Outputs: output, (h_n, c_n)</p>
    
    <ul>
    <li>output (seq_len, batch, hidden_size * num_directions): tensor containing the output features (h_t) from the last layer of the RNN, for each t. If a torch.nn.utils.rnn.PackedSequence has been given as the input, the output will also be a packed sequence.</li>
    <li>h_n (num_layers * num_directions, batch, hidden_size): tensor containing the hidden state for t=seq_len</li>
    <li>c_n (num_layers * num_directions, batch, hidden_size): tensor containing the cell state for t=seq_len</li>
    </ul>
  </blockquote>
  
  <p>It seems that the variables <code>output</code> and <code>h_n</code> both give the values of the hidden state. Does <code>h_n</code> just redundantly provide the last time step that's already included in <code>output</code>, or is there something more to it than that?</p>