Feature selection by overfitting a small sample size
Congratulations! You have suggested independently the Wrapper method for feature selection. Yes, you can use this method. 
However, consider that the wrapper method is slow since you have to train a model for each iteration. More than that, feature selection is a NP-complete problem so don't expect the optimal subset.