When can embeddings be useful for small input spaces?
The advantages of embedding is converting categorical variables to numeric variables. Those numerical variables have semantic meaning. Thus any valid mathematical operation for numerical values has interpretable meaning.
One-hot encoding just tells the model that two entities are different. Embedding tells how different the entities are.