Dimension mismatch while using Pytorch LSTM module
I have a pytorch pretrained model from which I am generating features/embedding for some input sentences. The features are essentially torch object. For example a sample input_embedding (a list of torch objects) for one sentence looks like below

[tensor([-0.8264,  0.2524], device='cuda:0', grad_fn=&lt;SelectBackward&gt;)]


Now, I want to pass this embedding through a custom model which is fundamentally a bi-directional LSTM:

def custom_model(input_embedding):
    #initialize BiLSTM
    bilstm = torch.nn.LSTM(input_size=1, hidden_size=1, num_layers=1, batch_first=False, bidirectional=True)
    #feed input to bilstm object
    bi_output, bi_hidden = bilstm(input_embedding)
    # more code ....
    return F.softmax(x)


I wanted to pass my input_embedding to this custom model to get the prediction output like below:

for item in input_embedding:
    y_pred = biLSTM_single_sentence_student_model(item)


But it is throwing error on the bi_output, bi_hidden = bilstm(input_embedding) line saying:

IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

Most likely I am not defining the bilstm object properly due to my lack of understanding of Pytorch nn.LSTM input.

Please suggest. 

The input to LSTM must be a 3d tensor of shape (seq_len, batch, input_size). In your example, you are basically providing a 2d tensor of shape seq_len, input_size as you mentioned, [-0.8264,  0.2524] is one example sentence.

So, you can modify your example as follows.

# a list of 2 sentences
input_embedding = [
    torch.FloatTensor([[-0.8264],  [0.2524]]),
    torch.FloatTensor([[-0.3259],  [0.3564]])
]
for item in input_embedding:
    # item is a 2d tensor of shape `seq_len, input_size`
    # so, we unsqueeze it to make it 3d `seq_len, 1, input_size` where batch_size = 1
    y_pred = custom_model(item.unsqueeze(1))
    print(y_pred.size()) # 2, 1, 2


Hope this helps!
