Feature selection and classification accuracy relation
feature selection involves several approaches just like methods for machine learning. Idea is to keep most relevant but not redundant feature for predictive model that can yield optimal accuracy.

In your case, I can not see which method you are using for feature selection but assuming that you are not taking account of multivariate nature of feature dependency. Say you have N features, likely reason that your model accuracy drops after n top feature(s) but improves by adding n+k (where n &lt; k &lt; N when features are in descending order based on information gain) is due to inter-dependency (more relevance and less redundancy) of top n and k features. Univariate feature selection does not necessarily get optimal model accuracy when features are inter-dependent and not mutually exclusive. From philosophical point of view, set of optimal features is analogous to a quote by Aristotle: "The whole is greater than the sum of its parts"!

For optimal feature selection, I often is Caret package in R language where one may do feature selection using recursive feature elimination (RFE) among several other approaches. There is also a package called mRMRe to do feature selection based on maximum relevance, minimal redundancy.

Best,
Samir