Output hidden state in OpenNMT-py
I just have a short question regarding the pytorch version of OpenNMT. There does not seem to be an option to return the hidden state of encoder and decoder in the options. Am I missing a flag or is this not an option in OpenNMT-py?

What do you mean by the encoder and decoder does not return hidden state? 

If you see the RNNEncoder, it returns encoder_final, memory_bank, lengths where the memory_bank represents the hidden state which is of shape seq_len x batch_size x hidden_size. And the encoder_final is in general used by the decoder in a sequence-to-sequence model.

Now, let's see the RNNDecoder. As we see, the forward() method returns a FlaotTensor and a dictionary of FlaotTensors.

(FloatTensor, dict[str, FloatTensor]):
* dec_outs: output from the decoder (after attn)
  ``(tgt_len, batch, hidden)``.
* attns: distribution over src at each tgt
  ``(tgt_len, batch, src_len)``.


Usually, we use the dec_outs in sequence-to-sequence tasks. For example, in natural language generation tasks, we feed the dec_outs to a softmax layer to predict tokens.

There are several other types of encoders/decoders which you can explore from the following two links.


https://github.com/OpenNMT/OpenNMT-py/tree/master/onmt/encoders
https://github.com/OpenNMT/OpenNMT-py/tree/master/onmt/decoders

