title: Scikit-learn feature selection for regression data
body: <p>I am trying to apply a univariate feature selection method using the Python module scikit-learn to a regression (i.e. continuous valued response values) dataset in svmlight format.</p>
  
  <p>I am working with scikit-learn version 0.11.</p>
  
  <p>I have tried two approaches - the first of which failed and the second of which worked for my toy dataset but I believe would give meaningless results for a real dataset.</p>
  
  <p>I would like advice regarding an appropriate univariate feature selection approach I could apply to select the top N features for a regression dataset. I would either like (a) to work out how to make the f_regression function work or (b) to hear alternative suggestions. </p>
  
  <p>The two approaches mentioned above:</p>
  
  <ol>
  <li>I tried using sklearn.feature_selection.f_regression(X,Y). </li>
  </ol>
  
  <p>This failed with the following error message:
  "TypeError: copy() takes exactly 1 argument (2 given)"</p>
  
  <ol>
  <li>I tried using chi2(X,Y). This "worked" but I suspect this is because the two response values 0.1 and 1.8 in my toy dataset were being treated as class labels?  Presumably, this would not yield a meaningful chi-squared statistic for a real dataset for which there would be a large number of possible response values and the number in each cell [with a particular response value and value for the attribute being tested] would be low? </li>
  </ol>
  
  <p>Please find my toy dataset pasted into the end of this message.</p>
  
  <p>The following code snippet should give the results I describe above.</p>
  
  <pre><code>from sklearn.datasets import load_svmlight_file
  
  X_train_data, Y_train_data = load_svmlight_file(svmlight_format_train_file) #i.e. change this to the name of my toy dataset file
  
  from sklearn.feature_selection import SelectKBest
  featureSelector = SelectKBest(score_func="one of the two functions I refer to above",k=2) #sorry, I hope this message is clear
  featureSelector.fit(X_train_data,Y_train_data)
  print [1+zero_based_index for zero_based_index in list(featureSelector.get_support(indices=True))] #This should print the indices of the top 2 features
  </code></pre>
  
  <p>Thanks in advance.</p>
  
  <p>Richard</p>
  
  <p>Contents of my contrived svmlight file - with additional blank lines inserted for clarity:</p>
  
  <p>1.8 1:1.000000 2:1.000000 4:1.000000 6:1.000000#mA</p>
  
  <p>1.8 1:1.000000 2:1.000000#mB</p>
  
  <p>0.1 5:1.000000#mC</p>
  
  <p>1.8 1:1.000000 2:1.000000#mD</p>
  
  <p>0.1 3:1.000000 4:1.000000#mE</p>
  
  <p>0.1 3:1.000000#mF</p>
  
  <p>1.8 2:1.000000 4:1.000000 5:1.000000 6:1.000000#mG</p>
  
  <p>1.8 2:1.000000#mH</p>