What is the correct way to use OHE lookup table for a pytorch RNN?
I am currently trying to build a LSTM RNN using pytorch. One input vector is represented as an array of 50 integers corresponding to a sequence of  at most 50 tokens with padding where each integer corresponds to an element from my vocabulary and the index of the 1 in the OHE vector. I want to have an embedding layer that just uses a lookup table to One-hot encode the integer-- kind of like tensorflow's OHE layer.  

Something like this "kind of" works 

import torch
import numpy as np
import torch.nn as nn 

# vocab_size is the number of words in your train, val and test set
# vector_size is the dimension of the word vectors you are using
vocab_size, vector_size = 5, 5
embed = nn.Embedding(vocab_size, vector_size)

# intialize the word vectors, pretrained_weights is a 
# numpy array of size (vocab_size, vector_size) and 
# pretrained_weights[i] retrieves the word vector of
# i-th word in the vocabulary
pretrained_weights = np.zeros((vocab_size, vector_size))
np.fill_diagonal(pretrained_weights, 1)
tmp =torch.from_numpy(pretrained_weights)
embed.weight = nn.Parameter(tmp,requires_grad=False )

# Then turn the word index into actual word vector
vocab = {"some": 0, "words": 1}
word_indexes = torch.from_numpy(np.array([vocab[w] for w in ["some", "words"]]))
word_vectors = embed(word_indexes)
word_vectors.data.numpy()


&gt;&gt;&gt;output
array([[1., 0., 0., 0., 0.],
       [0., 1., 0., 0., 0.]])


but it is very hacky, and doesn't play nicely with batches of input vectors.

What is the correct way to declare a OHE embedding layer at the begining of an RNN?

The method you are looking for is torch.nn.functional.one_hot. It was added to PyTorch later, so it's hard to find when you Google for it. 

 &gt;&gt;&gt; import torch
&gt;&gt;&gt; x = torch.arange(0,5)
&gt;&gt;&gt; x
tensor([0, 1, 2, 3, 4])
&gt;&gt;&gt; torch.nn.functional.one_hot(x)
tensor([[1, 0, 0, 0, 0],
      [0, 1, 0, 0, 0],
      [0, 0, 1, 0, 0],
      [0, 0, 0, 1, 0],
      [0, 0, 0, 0, 1]])

