choosing classifiers
See, your conclusions appear to be correct, your statistical results explain your conclusions , i would say statistically speaking even with a P-value of 0.055 you still have an error rate of 15 to 25% that you may be accepting a wrong null hypothesis and maybe your alternate hypothesis is right , so combining knowledge of statistics with data science i will suggest you to use catboost(if you have some categorical features in your dataset it will give you a strong baseline prediction before you start tuning it) otherwise random forest is good any day and if you have a small dataset (e.g. less than 1500 instances in your training data) i would suggest use random forest any day over MLP because MLP based classifiers need more data to classify better with less data they tend to become bias towards your training data.But if you have a large enough dataset you can actually use MLP based classifier try some basuc feature selection techniques such as RFECV or feature_importanece_ function you can use contained in every model after you train it . And if at the end of the day you are still confused to which model to use i would suggest use all of them stack them and you will have something better.Making these decisions solely on the basis of statistics is tough because statistics puts everything into an abstract form that is a whole dataset being represented by 1 number thus it is always prone to certain errors thus i always prefer to combine my data science experiments with my statistical results and then only conclude about my results.Also while using statistical analysis a lot of things depend on your dataset i.e. is it big enough to draw meaningful conclusions out of it what might be the sampling error and many more things .Hope this helps.