Am I overfitting my random forest model (more information in description)?
This is more like a comment than an answer but my newbie status doesn't allow me to comment yet.

You might be interested in reading this article:
Random forest

4 samples appears to me being low a number of samples,  I would recommend that you try using more samples and compare your results.

Besides, there are some good implementation of the algorithm out there, are you trying to re-implement it for academic purposes?