Is over fitting okay if test accuracy is high enough? I am trying to build a binary classifier. I have tried deep neural networks with various different structures and parameters and I was not able to get anything better than

Train set accuracy : 0.70102
Test set accuracy : 0.70001


Then I tried machine learning algorithms such as KNN and Decision Trees etc. And I found that Random forest Classifier from Scikit-learn with n_estimators=100 gave me

Train set accuracy : 1.0
Test set accuracy : 0.924068


I tried adjusting other parameters such as max_depth, criterion But the decrease in training set accuracy also caused the test set accuracy to drop. Like 

Train set accuracy : 0.82002
Test set accuracy : 0.75222


My question is, is this

Train set accuracy : 1.0
Test set accuracy : 0.924068


acceptable ? Even thought the model is over fitting, the test set accuracy is better.
If you properly isolate your test set such that it doesn't affect training, you should only look at the test set accuracy. Here are some of my remarks:


Having your model being really good on the train set is not a bad thing in itself. On the contrary, if the test accuracy is identical, you want to pick the model with the better train accuracy.
You want to look at the test accuracy. That is your primary concern. So pick the model that provides the best performance on the test set.


Overfitting is not when your train accuracy is really high (or even 100%). It is when your train accuracy is high and your test accuracy is low.
it is not abnormal that your train accuracy is higher than your test accuracy. After all, your model has an advantage with the train set since it's been given the correct answer already.



At the end of the day, training a machine learning model is like studying for a test. You (the model) use learning resources such as books, past exams, flash cards etc. (train set) to perform well on a test/exam (test set). Knowing your learning resources perfectly doesn't mean you are overfitting. You would be overfitting if this is all you knew and couldn't perform well on the exam at all.