Evaluating the performance of a random forest classifier
First of all, if you are trying to impute missing values with a RF model then take a look at the rfImpute() function.  

Second, your data in unbalanced, which is why your classification is not good, your model is biased towards the majority class (small) and so it classifies a lot of you cases into the majority class. The issue of imbalance needs to be addressed.

Validating is done with a test set, as the results you have obtained from the model are already done using Cross-Validation (known as OOB scores).