at the cost of slightly increasing the bias overall this produces better and more generalized model we will cover the bias-variance tradeoff in more detail in let' now use the random forest model to score and rank features based on their importance in [ ]from sklearn ensemble import randomforestclassifier rfc randomforestclassifier(rfc fit(bc_xbc_yout[ ]randomforestclassifier(bootstrap=trueclass_weight=nonecriterion='gini'max_depth=nonemax_features='auto'max_leaf_nodes=nonemin_impurity_split= - min_samples_leaf= min_samples_split= min_weight_fraction_leaf= n_estimators= n_jobs= oob_score=falserandom_state=noneverbose= warm_start=falsethe following code uses this random forest estimator to score the features based on their importance and we display the top most important features based on this score in [ ]importance_scores rfc feature_importances_ feature_importances [(featurescorefor featurescore in zip(bc_data feature_ namesimportance_scores)sorted(feature_importanceskey=lambda - [ ])[: out[ ][('worst area' )('worst radius' )('worst concavity' )('worst concave points' )('mean concave points' )('mean concavity' )('mean area' )('worst perimeter' )('worst texture' )('mean radius' )you can now use threshold based parameter to filter out the top features as needed or you can even make use of the selectfrommodel meta-transformer provided by scikit-learn by using it as wrapper on top of this model can you find out how many of the higher ranked features from the random forest model are in common with the previous two feature selectorsdimensionality reduction dealing with lot of features can lead to issues like model overfittingcomplex modelsand many more that all roll up to what we have mentioned as the curse of dimensionality refer to the section "dimensionality reductionin to refresh your memory dimensionality reduction is the process of reducing the total number of features in our feature set using strategies like feature selection or feature extraction we have already talked about feature selection extensively in the previous section we now cover feature extraction where the basic objective is to extract new features from the existing set of features such that the higher-dimensional dataset with many features can be reduced into lower-dimensional dataset of these newly created features very popular technique of linear data transformation from higher to lower dimensions is principal component analysisalso known as pca let' try to understand more about pca and how we can use it for feature extraction in the following sections