How to prune the features using recursive feature elimination? I have some kind of spatial data for nearly 1000 locations and at each location around 5000 features. I am doing neighborhood analysis to identify which features are predominant in local neighborhood.
I read that using recursive feature elimination only strong features can be kept and weak features can be eliminated. For some particular location I get neighborhood data X of numpy shape (14, 5000) where 5000 is features and 14 are local neighbors.
I also get target vector of 14 length having 3 class labels.
I tried to perform following things for pruning the features but getting value error. I am new to ML community and not sure whether doing the right things or not. Any suggestion will be helpful.
from sklearn.model_selection import StratifiedKFold
from sklearn.feature_selection import RFECV
from sklearn.ensemble import RandomForestClassifier

#X.shape (14, 5000) 
#target.shape (14,)
#len(np.unique(target)))  3 

rfc = RandomForestClassifier(random_state=101)
rfecv = RFECV(estimator=rfc, step=1, cv=StratifiedKFold(5), scoring='accuracy')
rfecv.fit(X, target)
print('Optimal number of features: {}'.format(rfecv.n_features_))

error
ValueError: n_splits=5 cannot be greater than the number of members in each class.
StratifiedKFold tries to keep balance between representation of classes in each fold. You have 5 folds and one or more of your classes have less than 5 instances in the data i.e. they can not be spread among 5 folds. So you can not use it.
That was the minor problem. ML-wise you have very few data points so the right way to cross-validate is leave-one-out.