How to tune parameters in Random Forest ? (Python Scikit Learn)
Question: 
    class sklearn.ensemble.RandomForestClassifier(n_estimators=10, criterion='gini', max_depth=None, min_samples_split=2, min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features='auto', max_leaf_nodes=None, bootstrap=True, oob_score=False, n_jobs=1, random_state=None, verbose=0, warm_start=False, class_weight=None)
    

I'm using a random forest model w/ 9 samples and about 7000 attributes. Of
these samples, there are 3 categories that my classifier recognizes.

I know this is far from ideal conditions but I'm trying to figure out which
attributes are the most important in feature predictions. **Which parameters
would be the best to tweak for optimizing feature importance?**

I tried different `n_estimators` and noticed that the amount of "significant
features" (i.e. nonzero values in the `feature_importances_` array) increased
dramatically.

I've read through the documentation but if anyone has any experience in this,
**I would like to know which parameters are the best to tune and a brief
explanation why.**


Answer: From my experience, there are three features worth exploring with the sklearn
RandomForestClassifier, in order of importance:

  * `n_estimators`

  * `max_features`

  * `criterion`

`n_estimators` is not really worth optimizing. The more estimators you give
it, the better it will do. 500 or 1000 is usually sufficient.

`max_features` is worth exploring for many different values. It may have a
large impact on the behavior of the RF because it decides how many features
each tree in the RF considers at each split.

`criterion` may have a small impact, but usually the default is fine. If you
have the time, try it out.

Make sure to use sklearn's [GridSearch](http://scikit-
learn.org/stable/modules/grid_search.html) (preferably GridSearchCV, but your
data set size is too small) when trying out these parameters.

If I understand your question correctly, though, you only have 9 samples and 3
classes? Presumably 3 samples per class? It's very, very likely that your RF
is going to overfit with that little amount of data, unless they are good,
representative records.

