High multi class classification with small data set I am working on a multi-class classifier for data set with 240K samples and ~1880 classes, the most  populated class is 4% of the dataset and large number of classes are less then 1% of the samples. 

I am trying to find a method to decide for which class I have enough information to correctly predict or to reject the prediction due to classifier performance for that particular class. 

Also, are there any good methods to deal with that kind of situations of high multi class with very low sample size?
Yeah. There have been scenarios like that. Imagenet has more than 1000 classes. Same could be said for certain cifar datasets. 

If deep learning, go with one-hot encoding for the labels.

Stick with something like multi-class AUC as metric instead of believing on Validation accuracy. 

Go with a keras DNN, then you could try advanced architectures, with the last Dense net to have 1880 neurons each representing one label. Also stick with categorical_crossentropy for loss.

Hope this helps.