Different number of features in train vs test when using Label Encoding This is not a duplicate of Different number of features in train vs test
There are some categorical columns in my data, and the cardinality for each of them is large, so I chose to use LabelEncoding over OneHotEncoding. However, some categories in the validation set do not appear in the training set, and they cause my model to perform very poorly. I was scraping the internet but couldn't find a way.
Is there any way I can use to solve this issue?
Thank you in advance.
To solve the high cardinality problem, follow the recommendation of Erwan. To solve the second issue you are mentioning, fit the OneHotEncoder on the train set and use the transformer on the test set, like this:
from sklearn.preprocessing import OneHotEncoder

enc = OneHotEncoder(handle_unknown = 'ignore')

enc.fit(train)

X_train = enc.transform(train)

X_test = enc.transform(test)