How to use embedding to reduce features for a regression problem I’m working on a regression problem in which I’d like to predict demand of different items. I have used holidays as a feature in my model, in a hot encoded format, i.e. I have 11 binary features each representing one holiday.
I’d like to reduce the size of my features and thinking to used embedding on the holiday features to represent them in lower dimensions.
I’m new to embedding. My question whether it make sense or not and if it does any hints on how to do it?
Doing it is easy. Simply create a mapping between your 11 values and embeddings of any size. Choosing the values for the embeddings is typically done through training a neural network that embeddings are part of. You could use dimensionality reduction techniques like PCA for instance as an alternative.
Now, embeddings only make sense if they represent a feature of dimensionality $D$ with embeddings of dimensionality between 1 and $D-1$. So in your case, I don't know how much you will benefit from the dimensionality reduction itself. You may experience benefits of replacing sparse with dense features, but this needs to be empirically tested.