What happens when the vocab size of an embedded layer is larger than the text corpus used in training? Full disclosure this question is based on following this tutorial: https://tinyurl.com/vmyj8rf8
I am trying to fully understand embedded layers in Keras. Imagine having a network to try and understand basic sentiment analysis as a binary classifier (1 positive sentiment and 0 negative sentiment). The toy dataset for this is as follows:
# Define 10 restaurant reviews
reviews =[
          'Never coming back!',
          'horrible service',
          'rude waitress',
          'cold food',
          'horrible food!',
          'awesome',
          'awesome services!',
          'rocks',
          'poor work',
          'couldn\'t have done better'
]#Define labels
labels = array([1,1,1,1,1,0,0,0,0,0])

This data can be used to train a really simple network as follows:
Vocab_size = 50
model = Sequential()
embedding_layer = Embedding(input_dim=Vocab_size,output_dim=8,input_length=max_length)
model.add(embedding_layer)
model.add(Flatten())
model.add(Dense(1,activation='sigmoid'))
model.compile(optimizer='adam',loss='binary_crossentropy',metrics=['acc'])
print(model.summary())

In order to feed this data into he network, we can one hot encode it using Keras one_hot as follows:
encoded_reviews = [one_hot(d,Vocab_size) for d in reviews]
print(f'encoded reviews: {encoded_reviews}')

We get the following output:
encoded reviews: [[14, 45, 43], [8, 2], [6, 43], [24, 1], [8, 1], [11], [11, 21], [16], [34, 40], [2, 25, 36, 15]]

I understand that the purpose of setting Vocab_size = 50, even though there are only around 20 unique words in the corpus is to give a large enough hashing space for the hashing algorithm behind one_hot to avoid collisions when the text is encoded.
If I train the model on these words (assume fixed length input and padding) and then get the weights of the embedded layer:
print(embedding_layer.get_weights()[0].shape)

(50, 8)

We can see this it is an array of 50 vectors that look like this as an example:
[ 0.17051394  0.13659576 -0.05245572 -0.12567708  0.06743167  0.05893507
-0.14506021  0.06448647]
My understanding is that each of these vectors corresponds to a word embedding for each word in the corpus. But if there are only 20 unique words in the corpus and Vocab_size is set larger than this then that can't be completely true? If Vocab_size &gt; corpus_vocab_size, then what do these embeddings represent? Any help would be appreciated.
tf.keras.layers.Embedding(..., embeddings_initializer=&quot;uniform&quot;*,..., *kwargs)


All the weights are initialized with the init strategy
All learn the optimum values  with the backprop
Weights for which there is no input will have zero output every time, hence no learning.
Hence these extra weights will remain at their initialization value

You may check these extra weights before and after.
weight = model.layers[0].get_weights() # Save before training
history = model.fit(x, y)

# These two should be same
weight[0][-1]  # Last weight - Before
model.layers[0].get_weights()[0][-1] # Last weight - After