I found the following very surprising.I trained different machine learning classifiers on a data set that includes $15$ attributes using $k$-fold cross validation. Then test on a different validation data set.The result were fine, accuracy was about $81\%$ and the f1-measure was about $83\%$.Out of curiosity, I tried to train and test on one single attribute, and I found that there is one attribute that outperform the previous result, accuracy was about $85\%$ and the f1-measure was about $86\%$.This results of the one attribute was always better than all attribute when using different binary classifiers.How common is this? Is there any interpretations for the reasons behind this?
binary classification: 1 attribute vs many
classification