title: Should Feature Selection be done before Train-Test Split or after?
body: <p>Actually, there is a contradiction of 2 facts that are the possible answers to the question:</p>
  
  <ol>
  <li><p>The conventional answer is to do it after splitting as there can be information leakage, if done before, from the Test-Set.</p></li>
  <li><p>The contradicting answer is that, if only the Training Set chosen from the whole dataset is used for Feature Selection, then the feature selection or feature importance score orders is likely to be dynamically changed with change in random_state of the Train_Test_Split. And if the feature selection for any particular work changes, then no Generalization of Feature Importance can be done, which is not desirable. Secondly, if only Training Set is used for feature selection, then the test set may contain certain set of instances that defies/contradicts the feature selection done only on the Training Set as the overall historical data is not analyzed. Moreover, feature importance scores can only be evaluated when, given a set of instances rather than a single test/unknown instance.</p></li>
  </ol>