the remaining features to obtain the new weights or scores this process is recursively carried out multiple times and each time features with the lowest scores/weights are eliminateduntil the pruned feature subset contains the desired number of features that the user wanted to select (this is taken as an input parameter at the startthis strategy is also popularly known as backward elimination let' select the top features on our breast cancer dataset now using rfe in [ ]from sklearn feature_selection import rfe lr logisticregression(rfe rfe(estimator=lrn_features_to_select= step= rfe fit(bc_xbc_yout[ ]rfe(estimator=logisticregression( = class_weight=nonedual=falsefit_intercept=trueintercept_scaling= max_iter= multi_class='ovr'n_jobs= penalty=' 'random_state=nonesolver='liblinear'tol= verbose= warm_start=false)n_features_to_select= step= verbose= we can now use the get_supportfunction to obtain the final selected features this is depicted in the following snippet in [ ]select_features_rfe rfe get_support(feature_names_rfe bc_data feature_names[select_features_rfeprint(feature_names_rfe['mean radius'mean texture'mean perimeter'mean smoothness'mean concavity'mean concave points'mean symmetry'texture error'worst radius'worst texture'worst smoothness'worst concavity'worst concave points'worst symmetry'worst fractal dimension'can we compare this feature subset with the one we obtained using statistical tests in the previous section and see which features are common among both these subsetsof course we canlet' use set operations to get the list of features that were selected by both these techniques in [ ]set(feature_names_kbestset(feature_names_rfeout[ ]{'mean concavity''mean perimeter''mean radius''mean texture''worst concave points''worst concavity''worst radius''worst texture'thus we can see that out of features are common and have been chosen by both the feature selection techniqueswhich is definitely interestingmodel-based selection tree based models like decision trees and ensemble models like random forests (ensemble of treescan be utilized not just for modeling alone but for feature selection these models can be used to compute feature importances when building the model that can in turn be used for selecting the best features and discarding irrelevant features with lower scores random forest is an ensemble model this can be used as an embedded feature selection methodwhere each decision tree model in the ensemble is built by taking training sample of data from the entire dataset this sample is bootstrap sample (sample taken with replacementsplits at any node are taken by choosing the best split from random subset of the features rather than taking all the features into account this randomness tends to reduce the variance of the model