How to properly use oversampling without inflating results?
First of all - have you tried to do modeling without oversampling? If you dont have imbalanced classes, why do you need oversampling? You dont always need tons of data to get a good model as a baseline.
Regarding should i apply the SMOTE only in train data - always leave some test data out, dont apply any transformation on the whole dataset (even normalization or missing values imputing). If you have some preprocessing - do it on train data set, train model. Then apply the preprocessing pipeline on test data and predict on it.
simplest models such as a 15-neuron MLP. - it doesnt sound as simplest solution to me. have you tried any other approaches? also, have you checked confusion matrix to see the classification error on test data for each of the class?