title: How to automate step forward feature selection in python?
body: <p>I have selected the data from Kaggle to practice random forest. <a href="https://www.kaggle.com/code/prashant111/random-forest-classifier-feature-importance/notebook" rel="nofollow noreferrer">https://www.kaggle.com/code/prashant111/random-forest-classifier-feature-importance/notebook</a>. I want to automate step-forward selection in python. Since there are numerous columns and I want to select columns that are between 4 to 10 to get the optimum result from minimum features. So, changing the value of <code>K_features</code> will be time-consuming and hectic so, I wanted to automate the process. But I am stuck in the process. Can somebody help me with this issue?</p>
  <pre><code>from mlxtend.feature_selection import SequentialFeatureSelector as sfs
  from sklearn.ensemble import RandomForestClassifier
  from sklearn.model_selection import train_test_split
  from imblearn.over_sampling import SMOTE
  
  y=df[' income_ &gt;50K']
  X=df.drop([' income_ &gt;50K'], axis=1)
  
  X_train, X_test, y_train, y_test = train_test_split(X,y, test_size=0.33, random_state=42)
  
  s=SMOTE(rnadom_state=42)
  X, y = s.fit_resample(X_train, y_train)
  
  scaler = StandardScaler()
  X_scaled = scaler.fit_transform(X)
  test_X_scaled = scaler.transform(X_scaled)
  
  rf_model = RandomForestClassifier()
  
  for i in range(3,15):
      test = sfs(rf_model, k_features=i, forward=False, floating=False, verbose=2, 
      scoring='accuracy', cv=3)
      test.fit(X_scaled, y)
  
      selected_cols = list(test.k_feature_idx_)
      df1 = df.iloc[:, selected_cols]```
  
  I wanted to iterate through 3 to 15 and calculate the accuracy and roc_score along with the feature name. 
  The result I wanted to see is like the below:
  
  | k_featur | accuracy       | roc score |selected_cols    |
  | -------- | -------------- | ----------|----------       |
  | 3        | 0.72           | 0.64      |[0,1,4,]         |
  | 4        | 0.78           | 0.74      |[0,1,4, 5]       |
  | 5        | 0.69           | 0.82      |[0,1,4,5,6]      |
  | 6        | 0.84           | 0.93      |[0,1,4,5,6,10]   |
  | 7        | 0.92           | 0.94      |[0,1,4,5,6,9,10] |
  and so on
  </code></pre>