Is there a particular order in which to do feature selection and sampling? I want to use feature selection and observation subsampling on my data, for several reasons:


feature selection for the usual motivations (reduce noise, decrease running time, etc.)
observation subsampling because I have strongly imbalanced data, and I want not to introduce bias towards the most prevalent class in downstream classifiers


My question is: is there a specific order in which I should do feature and observation selection? E.g. first feature selection then subsampling?
In my opinion, the methodological correct way to do it would be to first randomly select the observations (e.g., with stratified random sampling to sustain the class balances as in the original data) and then do any model building (feature selection is a part of model building) based on the sected examples.

You should asses the quality of your model with remaining observations that have not been used for either selecting features or training a model.

Hope this helps.