One Hot Encoding vs Word Embedding - When to choose one or another?
One-Hot Encoding is a general method that can vectorize any categorical features. It is simple and fast to create and update the vectorization, just add a new entry in the vector with a one for each new category. However, that speed and simplicity also leads to the &quot;curse of dimensionality&quot; by creating a new dimension for each category.
Embedding is a method that requires large amounts, both in the total amount of data and repeated occurrences of individual exemplars, and long training time. The result is a dense vector with a fixed, arbitrary number of dimensions.
They also differ at the prediction stage: a One-Hot Encoding tells you nothing of the semantics of the items; each vectorization is an orthogonal representation in another dimension. Embeddings will group commonly co-occurring items together in the representation space.
If you have enough training data, enough training time, and the ability to apply the more complex training algorithm (e.g., word2vec or GloVe), go with Embeddings. Otherwise, fall back to One-Hot Encoding.