title: SelectKBest sklearn varying computation times
body: <p>I was trying to do feature selection on a synthetic multilabel dataset. It was observed that the computation time for giving the complete dataset to SelectKBest was much higher as compared to time required while giving one feature at a time. In this example below only one label (or target variable) is considered.</p>
  
  <pre><code>import pandas as pd
  from sklearn.datasets import make_multilabel_classification
  from sklearn.feature_selection import chi2, SelectKBest, f_classif
  
  # Generate a multilabel dataset
  x, y = make_multilabel_classification(n_samples=40000, n_features = 1000, sparse = False, n_labels = 4, n_classes = 9,
    return_indicator = 'dense', allow_unlabeled = True, random_state = 1000)
  
  X_df = pd.DataFrame(x)
  y_df = pd.DataFrame(y)
  
  %%time
  selected_features2 = [] 
  for label in y_df.columns.tolist()[0:1]:
      selector = SelectKBest(f_classif, k='all')
      selected_features = [] 
      for ftr in X_df.columns.tolist():
          selector.fit(X_df[[ftr]], y_df[label])
          selected_features.extend(np.round(selector.scores_,4))
  </code></pre>
  
  <blockquote>
    <p>CPU times: user 3.2 s, sys: 0 ns, total: 3.2 s . Wall time: 3.18 s</p>
  </blockquote>
  
  <pre><code>%%time
  sel_features = []
  for label in y_df.columns.tolist()[0:1]:
      selector = SelectKBest(f_classif, k='all')
      selector.fit(X_df, y_df[label])
      sel_features.extend(np.round(selector.scores_,4))
  </code></pre>
  
  <blockquote>
    <p>CPU times: user 208 ms, sys: 37.2 s, total: 37.4 s Wall time: 37.4 s</p>
  </blockquote>
  
  <pre><code>%%time
  sel_features = []
  for label in y_df.columns.tolist()[0:1]:
      selector = SelectKBest(f_classif, k='all')
      selector.fit(X_df.as_matrix(), y_df[label].as_matrix())
      sel_features.extend(np.round(selector.scores_,4))
  </code></pre>
  
  <blockquote>
    <p>CPU times: user 220 ms, sys: 35.4 s, total: 35.7 s Wall time: 35.6 s .</p>
  </blockquote>
  
  <p>Why is there this much difference in the computation times?</p>