Initial embeddings for unknown, padding? Last time I've been passing pretrained word embeddings into LSTM to solve text classification problems. Usually, there are additional &lt;pad&gt;, &lt;unk&gt; replacements for padding and unknown types. Of course, there are no pretrained vectors for them.

Solutions I've come up with are


Fill them with random values
Fill them with zeros


Which approach is better/wrong? What is the common practice?

Note: I use pytorch+torchtext if it matters.
In my experience, what works well is :


For padding, fill a zero vector embedding (as pixel intensity in image data padding) is the only and best solution.
For words that don't have a pre-trained embedding, you should try to train them: as you do, fill them with random values when initializing, but set them to trainable.