<p dir="auto">Currently to load pretrained tensor into an Embedding I am running:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="embedding = torch.nn.Embedding(length, dim)
embedding.weight.data = pretrained_vectors_tensor"><pre class="notranslate"><span class="pl-s1">embedding</span> <span class="pl-c1">=</span> <span class="pl-s1">torch</span>.<span class="pl-s1">nn</span>.<span class="pl-v">Embedding</span>(<span class="pl-s1">length</span>, <span class="pl-s1">dim</span>)
<span class="pl-s1">embedding</span>.<span class="pl-s1">weight</span>.<span class="pl-s1">data</span> <span class="pl-c1">=</span> <span class="pl-s1">pretrained_vectors_tensor</span></pre></div>
<p dir="auto">This first initializes a tensor of  size <code class="notranslate">length x dim</code> which can be quite large and then throws all of that away. When creating the embedding there are two big tensors in memory which fill up my RAM, while there could be only one if there was a possibility to initialize weights from a parameter.</p>
<h3 dir="auto">Summary</h3>
<p dir="auto">Loading pre-trained embeddings is common practice with today's Embeddings. Most often than not, people use large pre-trained Embeddings such as Word2Vec, Glove or FastText with their models. As such there should be an easy and simple way of doing this so common operation.</p>
<h3 dir="auto">Current Behaviour</h3>
<p dir="auto">To load a pre-trained embedding one has to do the following:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="e = nn.Embedding(num_embedding, embedding_dim)
e.weight = nn.Parameter(pretrained_embedding)"><pre class="notranslate"><code class="notranslate">e = nn.Embedding(num_embedding, embedding_dim)
e.weight = nn.Parameter(pretrained_embedding)
</code></pre></div>
<p dir="auto">This involves converting a <code class="notranslate">Tensor</code> to <code class="notranslate">Variable</code> and access the class' arguments to override them.</p>
<h3 dir="auto">Suggested Behaviour</h3>
<p dir="auto">I propose doing this either with an explicit method, or an optional argument in the <code class="notranslate">__init__</code> function.</p>
<p dir="auto">This way you could either initialize an embedding as:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="e = nn.Embedding(num_embedding, embedding_dim, pretrained=pretrained_embedding)"><pre class="notranslate"><code class="notranslate">e = nn.Embedding(num_embedding, embedding_dim, pretrained=pretrained_embedding)
</code></pre></div>
<p dir="auto">or</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="e = nn.Embedding(num_embedding, embedding_dim)
e.load_pretrained(pretrained_embedding)"><pre class="notranslate"><code class="notranslate">e = nn.Embedding(num_embedding, embedding_dim)
e.load_pretrained(pretrained_embedding)
</code></pre></div>
<hr>
<p dir="auto">Let me know what you think. I'll do a PR as soon I have some feedback from you guys.</p>
<p dir="auto">Thank you</p>