Is it acceptable to use label encoding for nominal categorical data when one hot encoding would create too many features? I'm working on a short data science project to compare the accuracy of different classification methods. The groups decided to use and compare Random Forest, Naive Bayes and SVM.
The dataset we are using has four categorical features. Each of which has a large number of unique values.

There are 16537 unique combinations of 17370 unique values in
FeaureA.
There are 13860 unique combinations of 13852 unique  values
in FeaureB.
There are 3295 unique combinations of 29 unique  values
in FeaureC.
There are 1518 unique combinations of 29 unique  values
in FeaureD.

From what I've read the RF and NB algorithms should work fine with label encoding but SVM requires one hot encoding. However that would increase the number of features by ~35K. The performance cost seems like it would be significant. Ideally we would use the same encoding for all three algorithms. Would it be better to take the performance hit and try something like PCA for feature reduction?
Tree Based models like RF and XGBoost can handle the feature space created by either of the methods you suggested above. Additionally, you can try to do feature selection (use those statistical tests) to eliminate 0 variance features and then feed the filtered dataset to your models.