I am looking at this tutorial: https://www.dataquest.io/mission/75/improving-your-submissionAt section 8, finding the best features, it shows the following code.  import numpy as npfrom sklearn.feature_selection import SelectKBest, f_classifpredictors = [Pclass, Sex, Age, SibSp, Parch, Fare, Embarked, FamilySize, Title, FamilyId]# Perform feature selectionselector = SelectKBest(f_classif, k=5)selector.fit(titanic[predictors], titanic[Survived])# Get the raw p-values for each feature, and transform from p-values into scoresscores = -np.log10(selector.pvalues_)# Plot the scores.  See how Pclass, Sex, Title, and Fare are the best?plt.bar(range(len(predictors)), scores)plt.xticks(range(len(predictors)), predictors, rotation='vertical')plt.show()What is k=5 doing, since it is never used (the graph still lists all of the features, whether I use k=1 or k=all)?How does it determine the best features, are they independent of the method one wants to use (whether logistic regression, random forests, or whatever)?
How does SelectKBest work?
python;scikit learn
The SelectKBest class just scores the features using a function (in this case f_classif but could be others) and then removes all but the k highest scoring features. http://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.SelectKBest.html#sklearn.feature_selection.SelectKBestSo its kind of a wrapper, the important thing here is the function you use to score the features.For other feature selection techniques in sklearn read: http://scikit-learn.org/stable/modules/feature_selection.htmlAnd yes, f_classif and chi2 are independent of the predictive method you use. 