How to encode high cardinality categorical data? I have a dataset of 1600 rows and 28 columns. Only one column is partially complete with 1300 records. The rest is NaN. I did a value count of this columns and it has 84 different categories that are nominal. What is the best way to impute this column.
I need to convert these in numbers impute it and then convert back.
I understand that One-Hot encoding does not work in this case because of the high cardinality.
What is the best way to approach this problem?
You may want to look into Target Encoding as an example:
https://contrib.scikit-learn.org/category_encoders/targetencoder.html
Another post from the forums:
Problem with converting string to dummy variables