Appending a recurrent layer to PyTorch LSTM model with different hidden size
I'm developing a BI-LSTM model for sequence analysis using PyTorch. For which I am using torch.nn.LSTM. Using that module, you can have several layers with just passing a parameter num_layers to be the number of layers (e.g., num_layers=2). However all of them will have the same hidden_size which is partially fine for me, I just want to have all of them the same hidden_size but the last layer  with a different size. Basic example follows:

rnn = nn.LSTM(input_size=10, hidden_size=20, num_layers=2)
inp = torch.randn(5, 3, 10)
h0 = torch.randn(2, 3, 20)
c0 = torch.randn(2, 3, 20)
output, (hn, cn) = rnn(inp, (h0, c0))


The output dim is (5, 3,20)

One solution (But unfavorable to me) is implementing extra model that outputs the dimension I need and takes the input from the first model, e.g.,:

rnn_two = nn.LSTM(input_size=20, hidden_size=2)
output2, _ = rnn_two(output)


However, I do not wanna do this because I parallelize the model using DataParallel, so I need all to be one package. I was hoping to find something similar to keras, e.g.,: 

rnn.add(LSTM, hidden_size=2)


I have checked the LSTM source code but couldn't find what I need. 

Any suggestions?

If I'm not mistaken this can be done like this:

import torch.nn as nn
import torch.nn.functional as F

class RnnWith2HiddenSizesModel(nn.Module):
    def __init__(self):
        super(RnnWith2HiddenSizesModel, self).__init__()
        self.rnn = nn.LSTM(input_size=10, hidden_size=20, num_layers=2)
        self.rnn_two = nn.LSTM(input_size=20, hidden_size=2)

    def forward(self, inp, hc):
        output, _ = self.rnn(inp, hc)
        output2, _ = self.rnn_two(output)
        return output2


inp = torch.randn(5, 3, 10)
h0 = torch.randn(2, 3, 20)
c0 = torch.randn(2, 3, 20)

rnn = RnnWith2HiddenSizesModel()

output = RnnWith2HiddenSizesModel()(inp, (h0, c0))


tensor([[[-0.0305,  0.0327],
     [-0.1534, -0.1193],
     [-0.1393,  0.0474]],

    [[-0.0370,  0.0519],
     [-0.2081, -0.0693],
     [-0.1809,  0.0826]],

    [[-0.0561,  0.0731],
     [-0.2307, -0.0229],
     [-0.1780,  0.0901]],

    [[-0.0612,  0.0891],
     [-0.2378,  0.0164],
     [-0.1760,  0.0929]],

    [[-0.0660,  0.1023],
     [-0.2176,  0.0508],
     [-0.1611,  0.1053]]], grad_fn=&lt;CatBackward&gt;)

