What is projection layer in the context of neural machine translation using RNN?
I read a paper about machine translation, and it uses projection layer.
The projection layer is explained as follows: "Additional projection aims to reduce the dimensionality of the encoder output representations to match the decoder stack dimension."

Does anyone know this architecture or how to implement this layer in Pytorch?

The paper's link: https://www.aclweb.org/anthology/P18-1008.pdf

The model architecture:



It is a standard linear projection. You can just add nn.Linear(2 * model_dim,  model_dim) where model_dim is RNN dimension.

The encoder is bidirectional, with one RNNs in both directions having an output of dimension model_dim. The decoder only works in the forward direction, so it has states of only model_dim dimensions. It actually saves a lot of parameters in the multi-head attention because it makes the projection for keys and values only half size because they project from model_dim instead of 2 * model_dim.
