Choose-many categorical features: alternatives to one-hot encoding? I'm building a model to predict the lifetime value of a client based on the relational data we have on them.  The user table has a bunch of one-to-many child tables that might be predictive.  Grossly simplified, the child features boil down to things like:

a list of item categories that they've bought in the past
a list of the predominant colors in ads they've clicked on
etc, etc

In each case, the obvious feature comprises a list of ~ 0-10 choices from a categorical variable.  I have several of these features, some of which have as many as ~10k discrete values, so one-hot encoding would get very wide, very fast.
Aside: if there a term of art for this kind of &quot;list-of-tags feature&quot; that I'm referring to as &quot;choose many categorical&quot;, please tell me.
Question: Is there an dense encoding scheme that works with choose-many categorical features?
If your algorithm is based on gradient descent optimization, you can use embeddings, which are dense representation spaces for discrete elements.
Embeddings are supported by most deep learning frameworks such as pytorch or tensorflow.
Update: the fact that you want to have multiple discrete values does not prevent the possibility of using embeddings: you can just add all vectors together in a single value. The most straightforward approach for this would be to have a constant length for the list (equal to the maximum number of elements in all lists, or a sensible maximum value), filling with &quot;padding&quot; items the positions that are not needed. If you want to take the sequential appearance of the elements into account, instead of adding the vectors together you could apply convolutional layers or an LSTM over the embedded vectors.