Case when Out of Bag Error and Test error differs a lot in Random Forest I'm using random forest and the out of bag error for the level of one class is very different to its test error. I'm working with a cutt-of equal to c(0.2,0.8). Here's the case:

fmla &lt;- as.formula(paste("ex ~ ", paste(names(muestra.fullarbol[,-c(1,2,3,9,10,11,12,17,19,20,21,22,23,24,29,31,32,33,34,35,36,47)]), collapse= "+")))
&gt; bosque &lt;- randomForest(fmla , data=muestra.fullarbol ,mtry=12, ntree=1000  , cutoff=c(0.2,0.8),importance=TRUE)
&gt; bosque

Call:
 randomForest(formula = fmla, data = muestra.fullarbol, mtry = 12,      ntree = 1000, cutoff = c(0.2, 0.8), importance = TRUE) 
               Type of random forest: classification
                     Number of trees: 1000
No. of variables tried at each split: 12

        OOB estimate of  error rate: 15.81%
Confusion matrix:
     No Si class.error
No 3999  1     0.00025
Si  758 42     0.94750


As we see, the out of bag error for the level "SI" is 0.94750. However If I use the test set to get a sense of the error, the result is very different:

res.arbol &lt;- predict(bosque,test.fullarbol,type="class")
&gt; summary(res.arbol)
  No   Si 
7761   43 
&gt; table(res.arbol,test.fullarbol$ex)

res.arbol   No   Si
       No 6937  824
       Si    4   39
&gt; prop.table(table(res.arbol,test.fullarbol$ex),1)

res.arbol         No         Si
       No 0.89382811 0.10617189
       Si 0.09302326 0.90697674


Now we see thae error rate for the test set in the class "Si" is very low ,  it's equal to 0.093 and It doesn´t make sense to me.

I guess that the cut off is working just to predict out of the sample(muestra.fullarbol), but I'm not sure. What can be the reason of that huge difference?
What type of sampling method did you use to split the data? I'm guessing, by looking at the ratio of classes in your OOB confusion matrix and your test set confusion matrix that it wasn't stratified, is this correct? For instance, the class ratio in the OOB data is 4000:800 whereas in the test set it is 7761:43. As a starting point, you may want to split your data using stratified sampling so that the classes are fairly represented (in a statistical sense) in both your training data and test data. The ratio of classes in the test set should be identical or almost the same as the ratio used in the training data.

I think this would be a good starting point to figuring out the problem.